Bradford W. Mott

dblp:60/917 · DBLP profile ↗
← Back
136ranked-venue papers
10as first author
65since 2021 · last 2026
0000-0003-3303-4699ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Human-computer interaction and ubiquitous computing · 104 · 9 first-author · 52 since 2021Applied, interdisciplinary, general and emerging computing · 50 · 2 first-author · 15 since 2021Graphics, computer vision, multimedia, augmented reality and games · 32 · 6 first-author · 18 since 2021Artificial intelligence and machine learning · 17 · 1 first-author · 9 since 2021Software engineering, systems software and programming languages · 1Databases, data management, data science and information retrieval · 1
YearPublicationVenuePosition
2026 Co-Designing Unplugged Learning Activities with K-2 Teachers for Early AI Literacy Education
abstract
Introducing AI concepts in the earliest years of schooling can help children make sense of intelligent technologies, yet few resources exist for K–2 classrooms. This paper presents the design and outcomes of a professional development (PD) program supporting K–2 teachers as they explored AI literacy and co-designed unplugged classroom activities. Grounded in AI4K12's Five Big Ideas in AI framework, the PD combined hands-on learning, collaborative design, and micro-teaching opportunities. Guiding activities included Train the AI (pattern recognition), What Happens Next? (consequences of AI use), Who Did the Robot Hear? (data diversity), and Teach the Robot (model training). Educators then created screen-free, English Language Arts-aligned activities using storytelling, sorting, and embodied play to introduce AI topics such as machine learning and fairness in age-appropriate ways. The PD emphasized integrating AI into existing K–2 literacy routines, lowering implementation barriers while supporting vocabulary development, reasoning, and empathy. Teacher reflections revealed growing confidence in adapting AI topics for young learners and highlighted the value of peer collaboration, clear language, and tactile materials.
Jessica Vandenberg, Keisha Bailey, Claire Aguiar, Cecilia Xuning Zhang, Danny Schmidt, Treshonda Rutledge, Bradford W. Mott, Joseph P. Wilson
AAAI7
2026 From Embeddings to Chatbots: Playful NLP Activities for Middle School AI Literacy
abstract
As large language models (LLMs) and chatbots become increasingly prevalent, there is an urgent need to create engaging, age-appropriate learning activities that foster foundational AI literacy with a focus on natural language processing (NLP). This paper presents the iterative design and implementation of three instructional activities that introduce middle school learners (ages 11--14) to NLP concepts through playful, hands-on experiences aligned with the AI4K12 Big Idea of Natural Interaction. These activities include: (1) an unplugged card game that develops students' understanding of embeddings and similarity, (2) an unplugged collaborative sentence-generation challenge that illustrates how language models work, and (3) a web-based educational game in which students design and interact with chatbots. Each activity was implemented and refined across multiple educational contexts, including teacher professional development workshops, summer camps, and classroom implementations. All activities are designed to be easy to set up, requiring only commonly available classroom technology (e.g., laptops) and a few inexpensive materials (e.g., decks of cards), and are supported with facilitation guides and reflection prompts. Early implementations revealed areas for refinement, leading to clearer scaffolding that helped students connect gameplay to underlying NLP concepts, and post-refinement surveys indicated that students found the activities both enjoyable and educational. Findings suggest that blending unplugged and digital formats enhances comprehension, and that tailoring content to students' local contexts supports engagement. By making these activities openly available, this work contributes to the growing ecosystem of K–12 AI education resources and offers practical guidance for integrating NLP concepts into classroom instruction.
Jessica Vandenberg, Alex Goslen, Claire Aguiar, Wookhee Min, Veronica Cateté, Bradford W. Mott
AAAI6
2026 An Explainable AI Assistant for Introductory Programming Education: Improving Feedback Reliability with Instructor-AI Collaboration
Muntasir Hoq, Griffin Pitts, Bradford W. Mott, Seung Y. Lee, Jessica Vandenberg, Shuyin Jiao, Narges Norouzi, James C. Lester, Bita Akram
AIED (1)3
2026 Making AI Planning Visible: Narrative-Centered Goal-Directed AI Reasoning to Foster AI Literacy
Bradford W. Mott, Jessica Vandenberg, Srijita Chakraburty, Anne T. Ottenbreit-Leftwich, Emma Braaten, Cindy E. Hmelo-Silver, Amy Hutchison, Krista D. Glazewski, James C. Lester
AIED (3)1
2026 Introducing Adolescents to the Social Dimensions of AI Through Story-Driven Game-Based Learning
Jessica Vandenberg, Bradford W. Mott, Carlos Penilla, James C. Lester, Elizabeth Ozer
AIED (6)2
2026 Generating Clue-Driven Investigative Game Narratives with Large Language Models
Vikram Kumaran, Andy Smith, Wookhee Min, Randall Spain, Bradford W. Mott, James C. Lester
FDG5
2026 A Theory-Informed Narrative-Centered Model to Foster AI Literacy and Biomedical Career Interest
abstract
As artificial intelligence (AI) becomes increasingly central to healthcare and biomedical research, there is a growing need for learning experiences that help students understand AI concepts while envisioning future career pathways. This paper presents a theory-informed narrative-centered model designed to foster AI literacy and support emerging interest in biomedical careers among early adolescents aged 11-14. Grounded in narrative-centered learning and social cognitive theory, the model articulates how narrative structure, role-based engagement, and consequential decision making can support self-efficacy development, conceptual understanding, and interest in AI-enabled biomedical work. Building on this framework, we describe the design of a narrative-centered educational game in which learners assume the role of a medical intern and investigate patient cases using AI diagnostic tools. We then report findings from a pilot study with 25 students who played the game and participated in focus groups, drawing on gameplay trace data, in-game reflections, and qualitative feedback. Findings suggest high engagement, productive use of AI tools, and evidence of increased awareness of biomedical applications of AI, along with indications of perceived understanding of AI concepts. Together, the theoretical model, game design, and pilot findings illustrate how narrative-centered educational games can serve as research platforms while providing insight into how youth reason about AI in career-connected learning contexts.
Bradford W. Mott, Jessica Vandenberg, Carlos Penilla, Renee Navarro, James C. Lester, Elizabeth Ozer
FDG1
2026 Play, Explore, Reward: Introducing Reinforcement Learning Concepts through Game-Based Learning in Middle School
Veronica Cateté, Deniz Ozturk, Claire Aguiar, Jessica Vandenberg, Wookhee Min, Bradford W. Mott
ITiCSE (1)6
2026 Confidence Outpaces Interest: Upper Elementary AI Attitudes in a Science-Integrated Unit
Jessica Vandenberg, Bradford W. Mott, Srijita Chakraburty, Anne T. Ottenbreit-Leftwich, Cindy E. Hmelo-Silver, Krista D. Glazewski, James C. Lester
ITiCSE (2)2
2026 Collaborative Sensemaking of Unplugged AI in K-2 Language Arts: A Teacher Case Study
Jessica Vandenberg, Bradford W. Mott, Treshonda Rutledge, Cecilia Xuning Zhang, Joseph P. Wilson
ITiCSE (2)2
2026 INSIGHT: An Explainable, Instructor-Guided AI Assistant for Active Learning in CS1
abstract
Active learning in introductory programming depends on frequent, high-quality feedback, yet instructors often struggle to deliver consistent support at scale. In this poster, we introduce INSIGHT, an AI-driven classroom assistant designed to promote active learning in introductory programming (CS1) courses through scalable, personalized, and explainable feedback. The assistant combines the generative capabilities of large language models (LLMs) with instructor-in-the-loop authoring and an explainable code analysis engine to ensure pedagogically aligned support. Instructors can co-design problems with LLM assistance, provide exemplar solutions, define common student errors, and author targeted feedback. The AI engine analyzes student code submissions, identifies misconceptions, and maps them to instructor-verified feedback in real time. INSIGHT is designed to ensure that key educational concepts and common misconceptions are explicitly addressed by instructors, while also leveraging LLMs to provide reasonable feedback for novel or edge-case solutions that instructors may not have anticipated. By combining instructor expertise with the flexibility of generative AI, the assistant helps close feedback gaps and ensures more comprehensive coverage of student learning needs, especially in large or diverse classrooms with limited instructional support.
Muntasir Hoq, Jessica Vandenberg, Seung Y. Lee, Bradford W. Mott, James C. Lester, Narges Norouzi, Shuyin Jiao, Bita Akram
SIGCSE (2)4
2026 AI Meets Storytime: Co-Designing Unplugged K-2 ELA-Aligned AI Lessons with Teachers
abstract
Early elementary teachers are eager to introduce artificial intelligence (AI) concepts to their students but lack age-appropriate resources for exploring abstract ideas such as bias, trust, and fairness. This poster reports on a professional development (PD) program that engaged K-2 teachers in co-designing unplugged, literacy-aligned lessons for AI literacy in resource-constrained classrooms. Grounded in the AI4K12 ''Five Big Ideas in AI,'' the PD combined three phases: concept exploration, lesson co-design, and micro-teaching practice. Through the co-design process, teachers created lessons to help translate abstract AI concepts into classroom-friendly activities. Teachers emphasized the importance of simplifying technical language, embedding activities within familiar English Language Arts (ELA) routines, and using embodied and narrative approaches to sustain student engagement. This work contributes a replicable PD model for introducing AI to early elementary teachers, concrete examples of unplugged AI literacy lessons, and early evidence that abstract AI concepts can be meaningfully connected to literacy and play in K-2 education.
Jessica Vandenberg, Bradford W. Mott, Treshonda Rutledge, Claire Aguiar, Keisha Bailey, Cecilia Xuning Zhang, Joseph P. Wilson
SIGCSE (2)2
2026 Toward Equitable Collaboration in Elementary CS Education: Teacher Perspectives on Programming Models
abstract
Collaboration is central to computer science (CS) learning, yet little is known about how different collaborative programming models support upper elementary students. Although pair programming with a single computer is well established, younger learners often struggle with enacting driver–navigator roles. Networked environments enable two-computer programming models, where each student uses their own device, with or without structured roles. This poster presents early teacher insights on the feasibility of such models in classroom settings. Teachers highlighted the promise of using two-computers with roles for promoting equitable participation and sustained engagement, while noting practical challenges such as role design, switching, and device availability. Findings underscore the importance of aligning collaborative structures with classroom realities and providing scaffolds and routines to support facilitation, offering implications for both research and classroom practice in elementary CS education.
Jessica Vandenberg, Bradford W. Mott, Arif Rachmatullah, Satabdi Basu
SIGCSE (2)2
2026 Bridging Computational Thinking, Science, and Storytelling: Reflections on an Interdisciplinary Learning Approach
abstract
Integrating computational thinking (CT) into early K-12 education is increasingly recognized as essential for preparing students to navigate a technology-driven world. Digital storytelling, with its capacity to combine narrative expression with programming, offers a promising interdisciplinary strategy for promoting CT. This experience report presents a narrative-centered learning environment that integrates digital storytelling, block-based programming, and hands-on maker activities to foster CT and interdisciplinary learning in upper elementary classrooms. Grounded in a problem-based storyline, the environment engages students in solving real-world-inspired challenges through physical science experimentation and interactive narrative creation. We describe a multi-week classroom implementation with fourth- and fifth-grade students and analyze survey responses and programming artifacts from 41 participants to explore how CT concepts and practices, including sequencing, conditionals, and debugging, emerged in their work. While students demonstrated statistically significant gains in CT, they also encountered challenges related to narrative coherence, science alignment, and conditional logic. We reflect on what did and did not work, offering design insights for educators and designers adopting interdisciplinary, story-driven approaches to computing education.
Jessica Vandenberg, Andy Smith, Robert Monahan, James Minogue, Kevin M. Oliver, Aleata Hubbard Cheuoua, Cathy Ringstaff, Bradford W. Mott
SIGCSE (1)8
2025 Shaping AI Interest in Rural Middle Schools with Unplugged Learning: Gender Differences and Teacher Insights
abstract
Adoption of artificial intelligence (AI) is at an inflection point. With daily use of AI escalating due to widely available software tools, educators, researchers, and policymakers must adapt swiftly to changing educational needs. While think tanks and Big Tech companies often promote the notion that AI serves as a powerful tool for democratizing access to knowledge and opportunities, our work in rural communities underscores the disparity in access to AI education and related opportunities. In this paper, we report on our experience introducing foundational AI concepts to rural middle school students using an unplugged game-based learning activity. By providing engaging learning experiences to rural populations, we hope to broaden interest in and understanding of AI technologies. To this end, we conducted a classroom study in which two middle school teachers implemented our unplugged AI learning activity with their students. Analyzing survey data from 60 of the participating students, we explore the impact of the activity on their interest in AI, their conceptual understanding, and examine potential gender differences. Additionally, we share insights from the teachers who participated in our professional development sessions in preparation for the classroom implementations.
Hansol Lim, Danielle Boulden, Jessica Vandenberg, Veronica Cateté, Wookhee Min, Bradford W. Mott
AAAI6
2025 Predicting Facilitator Interventions in Collaborative Game-Based Learning with Student Dialogue Analysis
Priyanka Khare, Halim Acosta, Dan Carpenter, Haesol Bae, Bradford W. Mott, Seung Y. Lee, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
AIED (3)6
2025 Fostering AI Literacy Through Strategic Play: A Competitive Pathfinding Game for Middle School
Claire Aguiar, Dan Carpenter, Jessica Vandenberg, Wookhee Min, Veronica Cateté, Bradford W. Mott
CoG6
2025 AI4Health: A Narrative-Centered Educational Game for AI-Infused Biomedical Career Exploration
abstract
As AI becomes increasingly central to healthcare and biomedical research, there is a growing need for tools that introduce students to AI in engaging, authentic contexts. This paper presents AI4Health, a narrative-centered educational game that places middle school students in the role of a medical intern investigating real-world health mysteries. In the game's first episode, players gather evidence, interview characters, and use machine learning models with different training data and performance characteristics. Through branching dialogue and contextual problem-solving, players explore the impact of training data on model reliability and consider the ethical implications of AI-assisted diagnosis. This paper showcases the core mechanics, dialogue system, and interactive narrative structure that support AI literacy and health career exploration, and highlights plans to extend the game through additional biomedical scenarios and classroom integration.
Bradford W. Mott, Jessica Vandenberg, Carlos Penilla, Sean Hennigan, James C. Lester, Elizabeth Ozer
CoG1
2025 Designing a Narrative-Centered Game to Promote AI Literacy and Health Career Exploration
abstract
As artificial intelligence (AI) continues to transform industries across society, it is having a profound impact on healthcare and biomedical research. To prepare students for this evolving landscape, there is a growing need for learning experiences that build foundational AI literacy and connect to real-world career pathways. However, most middle school students lack access to engaging and personally meaningful opportunities in AI education and career exploration. NarrativeCentered Learning (NCL) offers powerful affordances for contextualizing AI literacy through engaging storylines and role-based problem-solving. In parallel, Social Cognitive Career Theory (SCCT) emphasizes how students' beliefs about their abilities, expectations about outcomes, and personal goals influence the development of their academic and career trajectories. This paper introduces a design framework that integrates NCL and SCCT to foster both AI literacy and health-related career interest. We apply this framework to the design of a narrative game in which students take on the role of a medical intern investigating virtual patient cases using AI tools. We report findings from a usability study with 25 middle school students who played the game's first episode and participated in structured focus groups. Student feedback suggests that the game supported engagement, sparked curiosity, and encouraged emerging career interest. These findings offer preliminary support for the framework and inform the design of career-connected AI learning.
Bradford W. Mott, Jessica Vandenberg, Carlos Penilla, Sean Hennigan, Renee Navarro, James C. Lester, Elizabeth Ozer
CoG1
2025 A Multi-View Predictive Student Modeling Framework with Interpretable Causal Graph Discovery for Collaborative Learning Analytics
Halim Acosta, Seung Y. Lee, Daeun Hong, Wookhee Min, Bradford W. Mott, Cindy E. Hmelo-Silver, James C. Lester
EDM5
2025 Automated Identification of Logical Errors in Programs: Advancing Scalable Analysis of Student Misconceptions
Muntasir Hoq, Ananya Rao, Reisha Jaishankar, Krish Piryani, Nithya Janapati, Jessica Vandenberg, Bradford W. Mott, Narges Norouzi, James C. Lester, Bita Akram
EDM7
2025 Collaborative Game-based Learning Analytics: Predicting Learning Outcomes from Game-based Collaborative Problem Solving Behaviors
Halim Acosta, Daeun Hong, Seung Y. Lee, Wookhee Min, Bradford W. Mott, Cindy E. Hmelo-Silver, James C. Lester
LAK5
2025 "Like a GPS": Analyzing Middle School Student Responses to an Interactive Pathfinding Activity
abstract
Engaging middle school students in complex computational topics such as AI can pose unique challenges to educators, ranging from simplifying potentially difficult mathematical material to maintaining student interest in the subject. One approach to help address these challenges is to utilize hands-on, real-world examples. We conducted a week-long summer camp for 24 students, centering each day around activities aligned with one of the Five Big Ideas in AI. Students participated in exit ticket reflections following each activity. The pathfinding activities, which occurred on one of the days, incorporated real-world examples and digital simulations of three pathfinding algorithms (breadth-first search, depth-first search, and A*), including an activity modeled after the game Pac-Man. Thematic analysis of exit-ticket responses revealed four major themes regarding students' key takeaways from the activities: (1) pathfinding for character movement, notably in video games like Minecraft; (2) pathfinding as a means of efficient navigation; (3) theoretical reasoning regarding the speed of the A* algorithm compared to others, highlighting its intelligent search mechanism; and (4) empirical reasoning based on personal experience during activities, where some students noted A* consistently performed fastest. These findings indicate that students not only engaged with AI concepts but also demonstrated a nuanced understanding of algorithmic efficiency. We examine the implications of these findings on understanding student engagement with interactive pathfinding activities and highlight the potential for future work in this area.
Claire Aguiar, Dan Carpenter, Jessica Vandenberg, Alex Goslen, Wookhee Min, Veronica Cateté, Bradford W. Mott
SIGCSE (2)7
2025 Introducing Reinforcement Learning Concepts to Middle School Students with Game-Based Learning
Matthew Presson, Anisha Gupta, Jessica Vandenberg, Alex Goslen, Wookhee Min, Veronica Cateté, Bradford W. Mott
SIGCSE (2)7
2025 Conceptualizing the Support and Learning of K-2 Educators around Artificial Intelligence in Language Arts
Jessica Vandenberg, Ryan Torbey, Cecilia Xuning Zhang, Bradford W. Mott, Keisha Bailey, Joseph P. Wilson
SIGCSE (2)4
2024 Supporting Upper Elementary Students in Learning AI Concepts with Story-Driven Game-Based Learning
abstract
Artificial intelligence (AI) is quickly finding broad application in every sector of society. This rapid expansion of AI has increased the need to cultivate an AI-literate workforce, and it calls for introducing AI education into K-12 classrooms to foster students’ awareness and interest in AI. With rich narratives and opportunities for situated problem solving, story-driven game-based learning offers a promising approach for creating engaging and effective K-12 AI learning experiences. In this paper, we present our ongoing work to iteratively design, develop, and evaluate a story-driven game-based learning environment focused on AI education for upper elementary students (ages 8 to 11). The game features a science inquiry problem centering on an endangered species and incorporates a Use-Modify-Create scaffolding framework to promote student learning. We present findings from an analysis of data collected from 16 students playing the game's quest focused on AI planning. Results suggest that the scaffolding framework provided students with the knowledge they needed to advance through the quest and that overall, students experienced positive learning outcomes.
Anisha Gupta, Seung Y. Lee, Bradford W. Mott, Srijita Chakraburty, Krista D. Glazewski, Anne T. Ottenbreit-Leftwich, J. Adam Scribner, Cindy E. Hmelo-Silver, James C. Lester
AAAI3
2024 Unplugged K-12 AI Learning: Exploring Representation and Reasoning with a Facial Recognition Game
abstract
With the growing prevalence of AI, the need for K-12 AI education is becoming more crucial, which is prompting active research in developing engaging and age-appropriate AI learning activities. Efforts are underway, such as those by the AI4K12 initiative, to establish guidelines for organizing K- 12 AI education; however, effective instructional resources are needed by educators. In this paper, we describe our work to design, develop, and implement an unplugged activity centered on facial recognition technology for middle school students. Facial recognition is integrated into a wide range of applications throughout daily life, which makes it a familiar and engaging tool for students and an effective medium for conveying AI concepts. Our unplugged activity, “Guess Whose Face,” is designed as a board game that focuses on Representation and Reasoning from AI4K12’s 5 Big Ideas in AI. The game is crafted to enable students to develop AI competencies naturally through physical interaction. In the game, one student uses tracing paper to extract facial features from a familiar face shown on a card, such as a cartoon character or celebrity, and then other students try to guess the identity of the hidden face. We discuss details of the game, its iterative refinement, and initial findings from piloting the activity during a summer camp for rural middle school students.
Hansol Lim, Wookhee Min, Jessica Vandenberg, Veronica Cateté, Bradford W. Mott
AAAI5
2024 Multimodal Learning Analytics for Predicting Student Collaboration Satisfaction in Collaborative Game-Based Learning
Halim Acosta, Seung Y. Lee, Bradford W. Mott, Haesol Bae, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
EDM3
2024 AI Planning is Elementary: Introducing Young Learners to Automated Problem Solving
abstract
Recent years have seen growing awareness of the need to advance AI literacy for K-12 students to empower them in understanding, evaluating, and using AI. Automated problem solving is a fundamental aspect of AI, enabling machines to mimic human problemsolving abilities. Fostering awareness and interest in AI capabilities such as automated problem solving should begin early, including in the elementary grades. Although AI planning can be a complex topic, leveraging the benefits of game-based learning offers a promising approach to engage young children in learning about this important AI concept. In this work, we explore the interactions and outcomes of upper elementary students (ages 8 to 11) playing a quest on AI planning embedded within a game-based learning environment. Results indicate that students experienced positive learning gains from pre-test to post-test, while analyzing trace data from the game provides insights into challenges students faced as they attempted the in-game missions.
Bradford W. Mott, Anisha Gupta, Jessica Vandenberg, Srijita Chakraburty, Anne T. Ottenbreit-Leftwich, Cindy E. Hmelo-Silver, J. Adam Scribner, Seung Y. Lee, Krista D. Glazewski, James C. Lester
ITiCSE (2)1
2024 Engaging Students from Rural Communities in AI Education with Game-Based Learning
abstract
As the presence of artificial intelligence (AI) technologies increases throughout everyday life, so does the need to engage rural communities in AI learning experiences, as these communities often have limited access to such educational opportunities. This work presents three game-based learning activities rooted in core AI concepts: natural language processing, search, and reinforcement learning. These activities were implemented in a summer camp with middle grades students in a rural area of the USA. We share an overview of the activities, as well as key observations and takeaways from student responses in post-activity surveys.
Alex Goslen, Anisha Gupta, Smrithi Muthukrishnan, Raven Midgett, Wookhee Min, Jessica Vandenberg, Veronica Cateté, Bradford W. Mott
SIGCSE (2)8
2024 Towards Attention-Based Automatic Misconception Identification in Introductory Programming Courses
abstract
Identifying misconceptions in student programming solutions is an important step in evaluating their comprehension of fundamental programming concepts. While misconceptions are latent constructs that are hard to evaluate directly from student programs, logical errors can signal their existence in students' understanding. Tracing multiple occurrences of related logical bugs over different problems can provide strong evidence of students' misconceptions. This study presents preliminary results of utilizing an interpretable state-of-the-art Abstract Syntax Tree-based embedding neural network to identify logical mistakes in student code. In this study, we show a proof-of-concept of the errors identified in student programs by classifying correct versus incorrect programs. Our preliminary results show that our framework is able to automatically identify misconceptions without designing and applying a detailed rubric. This approach shows promise for improving the quality of instruction in introductory programming courses by providing educators with a powerful tool that offers personalized feedback while enabling accurate modeling of student misconceptions.
Muntasir Hoq, Jessica Vandenberg, Bradford W. Mott, James C. Lester, Narges Norouzi, Bita Akram
SIGCSE (2)3
2024 Supporting Student Engagement in K-12 AI Education with a Card Game Construction Toolkit
abstract
With the growing prevalence of AI, the need for K-12 AI education is becoming more crucial, which is prompting active research in developing engaging AI learning activities. In this paper, we present our work on a game construction toolkit for middle school students and educators that enables them to tailor an AI-focused unplugged card game activity. In our prior work, we designed, developed, and piloted an unplugged card game activity where players predict the identity of a person based on hand-drawn features extracted from a set of facial cards. The activity aims to teach AI concepts aligned with one of the big ideas in AI utilizing techniques from facial recognition. During our pilot testing of the activity, we discovered that creating face cards that capture students' interest is a crucial factor in promoting student engagement. As a result, we designed a card game construction toolkit that allows students and educators to craft their own face card decks using photos that are personally interesting to them, looking to foster engagement and improve replayability of the activity. The toolkit's design is focused on ensuring easy accessibility and features a simple web-based interface that allows users to download and print their customized cards. We expect this toolkit will enhance the usability and educational effectiveness of our unplugged K-12 AI education activity.
Hansol Lim, Wookhee Min, Jessica Vandenberg, Veronica Cateté, Judith Uchidiuno, Bradford W. Mott
SIGCSE (2)6
2024 Procedural Level Generation in Educational Games From Natural Language Instruction
abstract
In the evolving field of mixed-initiative game design, where procedural content generation plays a pivotal role, establishing a comprehensive approach that empowers non-technical designers to actively shape content generation is essential. Recent developments in large language models significantly alter the landscape of automated text-based content generation. These models offer a significant advantage in mixed-initiative procedural level generation by providing designers with intuitive, natural language interfaces. The framework presented in this paper interprets natural language inputs, detailing level design constraints and optimization goals, to aid in the cooperative development of game levels for a strategy game aimed at environmental sustainability education. It enables designers to articulate their vision concerning the problem domain, goal metrics, and desired difficulty level through a textual description. By utilizing large language models, the framework extracts semantic constraints and optimization objectives, which are then used to generate candidate game levels. The efficacy of these levels is assessed by game-playing agents trained through advanced deep reinforcement learning methods, ensuring alignment with the designer's original specifications. We further evaluate our framework with both experts and non-experts in designing levels for our strategy game. Their detailed responses confirm that our framework effectively translates natural language descriptions into playable game levels, accurately capturing the designers' intended objectives.
Vikram Kumaran, Dan Carpenter, Jonathan P. Rowe, Bradford W. Mott, James C. Lester
IEEE Trans. Games4
2024 Exploring Gameplay and Learning in a Narrative-Centered Digital Game for Elementary Science Education
abstract
Recent years have seen increased exploration of the transformative potential of digital games for K-12 education. Narrative-centered digital games for learning integrate complex problem solving within compelling interactive stories. By leveraging the inherent structure of narrative and the engaging interactions afforded by commercial game engines, narrative-centered digital games for learning engage students in situated learning activities. This article presents details on the iterative design and development of a narrative-centered digital game for learning that focuses on science education for fifth-grade students. We then explore how student gameplay and learning relate by leveraging interaction log data from over 700 students playing the game. Specifically, we analyze student gameplay achievements using clustering and examine how gameplay and learning outcomes differ among the groups identified. Furthermore, we investigate if gender has an effect on student learning within the groups and what gender differences are found within the groups. The findings show that students who complete more quests and earn better in-game rewards achieve higher learning gains, and while differences exist in game playing characteristics between males and females the learning outcomes are similar.
Seung Y. Lee, Bradford W. Mott, Jessica Vandenberg, Hiller A. Spires, James C. Lester
IEEE Trans. Games2
2023 Enhancing Stealth Assessment in Collaborative Game-Based Learning with Multi-task Learning
Anisha Gupta, Dan Carpenter, Wookhee Min, Bradford W. Mott, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
AIED4
2023 End-to-End Procedural Level Generation in Educational Games with Natural Language Instruction
abstract
As the role of procedural content generation in mixed-initiative game design continues to grow, it is crucial to develop an end-to-end approach that enables non-technical designers to artfully guide content generation. Recent advances in large language models, such as GPT-4, are rapidly transforming the landscape of automated generation of text-based content. Large language models have significant potential for mixed-initiative procedural level generation by providing natural language interfaces for designers. This paper presents an end-to-end procedural level generation framework that interprets natural language descriptions of level design constraints and optimization objectives to facilitate the collaborative creation of game levels for a strategy game focused on environmental sustainability education. The framework enables designers to specify a problem domain, goal metrics, and target difficulty via natural language description. It then employs large language models for the semantic extraction of constraints and optimization targets to drive the generation of candidate levels. Generated game levels are evaluated via game-playing agents trained with deep reinforcement learning techniques to ensure the game levels meet the level designer’s specifications. Manual evaluation by authors shows that the proposed framework can effectively transform designers’ natural language descriptions into fully playable game levels that reflect their intended design objectives.
Vikram Kumaran, Dan Carpenter, Jonathan P. Rowe, Bradford W. Mott, James C. Lester
CoG4
2023 Examining the Relationship of Gameplay and Learning in a Narrative-Centered Digital Game for Science Education
abstract
Recent years have seen increased exploration of the transformative potential of digital games for K-12 education. Narrative-centered digital games for learning integrate complex problem solving within compelling interactive stories. By leveraging the inherent structure of narrative, narrative-centered digital games for learning engage students in situated learning activities. In this paper, we investigate how student gameplay and learning relate in a narrative-centered digital game for learning that focuses on science education for fifth grade students. Specifically, leveraging interaction log data from a study with over 700 students we analyze student gameplay achievements using clustering and examine how gameplay and learning outcomes differ among the groups identified. The findings show that students who complete more quests and earn better in-game rewards achieve higher learning gains.
Seung Y. Lee, Bradford W. Mott, Jessica Vandenberg, Hiller A. Spires, James C. Lester
CoG2
2023 Leveraging Game Design Activities for Middle Grades AI Education in Rural Communities
abstract
The ever pervasive nature of artificial intelligence (AI) in our world necessitates a focus on fostering an AI literate society. Young children, those aged 11 to 14, are at a critical point in developing their dispositions toward and perceptions of science, technology, engineering, and mathematics (STEM), which influences their future education and career interests. Youth in rural areas are in particular need of access to AI learning opportunities to prepare them for the future workforce; digital games may be one way to attract young, rural students to STEM education and careers. In this paper, we explore how to introduce rural middle grades students to foundational AI concepts through digital game design activities. To inform our efforts and to establish an understanding of what these student populations as well as their teachers know about AI and games, we conducted a set of interviews and focus groups. In brief, students’ awareness and understanding of AI varied significantly, whereas teachers had limited knowledge of AI. Moreover, students shared great interest in playing and designing games. In support of our findings, we are developing a set of game design activities around five core AI concepts and ensuring the activities are of interest to our rural students.
Jessica Vandenberg, Wookhee Min, Veronica Cateté, Danielle Boulden, Bradford W. Mott
FDG5
2023 Fostering Interdisciplinary Learning for Elementary Students Through Developing Interactive Digital Stories
Anisha Gupta, Andy Smith, Jessica Vandenberg, Rasha Elsayed, Kimkinyona Fox, James Minogue, Aleata Hubbard Cheuoua, Kevin M. Oliver, Cathy Ringstaff, Bradford W. Mott
ICIDS (2)10
2023 Integrating Storytelling and Making: A Case Study in Elementary School
Robert Monahan, Jessica Vandenberg, Andy Smith, Anisha Gupta, Kimkinyona Fox, Rasha Elsayed, Aleata Hubbard Cheuoua, James Minogue, Kevin M. Oliver, Cathy Ringstaff, Bradford W. Mott
ICIDS (2)11
2023 Multimodal CS Education Using a Scaffolded CSCL Environment
abstract
There is a growing need for 21st-century workers to be digitally literate and to possess computational thinking and collaborative problem-solving skills. Computer-supported collaborative learning (CSCL) focused on computational thinking can guide students toward the co-development of these skills. In this work, we present our approach to integrating virtual and physical learning modalities into InfuseCS, a CSCL environment. InfuseCS uses problem-based learning scenarios to situate upper elementary school students (ages 8 to 11) in a CSCL setting to foster their computational thinking and science knowledge construction as they collaborate to create digital narratives.
Robert Monahan, Jessica Vandenberg, Anisha Gupta, Andy Smith, Rasha Elsayed, Kimkinyona Fox, Aleata Hubbard Cheuoua, Cathy Ringstaff, James Minogue, Kevin M. Oliver, Bradford W. Mott
ITiCSE (2)11
2023 Fostering Upper Elementary AI Education: Iteratively Refining a Use-Modify-Create Scaffolding Progression for AI Planning
abstract
The growing ubiquity of artificial intelligence (AI) is reshaping much of daily life. This in turn is raising awareness of the need to introduce AI education throughout the K-12 curriculum so that students can better understand and utilize AI. A particularly promising approach for engaging young learners in AI education is game-based learning. In this work, we present our efforts to embed a unit on AI planning within an immersive game-based learning environment for upper elementary students (ages 8 to 11) that utilizes a scaffolding progression based on the Use-Modify-Create framework. Further, we present how the scaffolding progression is being refined based on findings from piloting the game with students.
Bradford W. Mott, Anisha Gupta, Krista D. Glazewski, Anne T. Ottenbreit-Leftwich, Cindy E. Hmelo-Silver, J. Adam Scribner, Seung Y. Lee, James C. Lester
ITiCSE (2)1
2023 "AI Teaches Itself": Exploring Young Learners' Perspectives on Artificial Intelligence for Instrument Development
abstract
Children encounter and use artificial intelligence (AI) with regularity, but the depth of their understanding of AI is often limited. In service of growing an AI and technology-literate K-12 population, it is important for young learners to engage in AI learning activities early and often. To foster the design of AI curricula, it is essential to understand what young children already know and how they feel about AI. The nascent field of AI-related self-report instrument development focuses largely on adult populations or AI's use in specific contexts, such as medicine. There remains a critical need to develop an AI attitudinal survey for young learners (ages 9 to 11). Building upon the extant survey development work of those in education and AI, we have designed a brief survey on students' self-efficacy for AI, interest and motivation toward AI, and attitudes toward AI. We used cognitive interviewing processes to ensure the items in the survey were readable and understandable by young students. Preliminary findings indicate young students have mixed understanding of what AI is, what it can do, and how they feel about AI. We discuss implications for researchers and practitioners and provide an overview of our continuing efforts to validate this instrument.
Jessica Vandenberg, Bradford W. Mott
ITiCSE (1)2
2023 Toward AI-infused Game Design Activities for Rural Middle Grades Students
abstract
The ubiquity of artificial intelligence (AI) in everyday life suggests the need to ensure young students know about AI, its uses and limitations, and its benefits and risks, while enabling them to develop expertise in using AI-driven technologies. To support rural middle grades students and educators in learning and teaching AI concepts, we are designing AI-focused learning activities centered around the creation of digital gameplay experiences. To inform our designs, we conducted educator interviews and student focus groups to gain insights into their understanding of AI, their computer science background, and their knowledge and interest in gaming. Building on findings from these interviews and focus groups, we have designed a set of hands-on activities to elicit deeper feedback from students and educators on their preferences, points of confusion, and interests. In this work, we present our initial AI-infused game design activities.
Jessica Vandenberg, Wookhee Min, Anisha Gupta, Veronica Cateté, Danielle Boulden, Bradford W. Mott
ITiCSE (2)6
2023 Effects of Modalities in Detecting Behavioral Engagement in Collaborative Game-Based Learning
abstract
Collaborative game-based learning environments have significant potential for creating effective and engaging group learning experiences. These environments offer rich interactions between small groups of students by embedding collaborative problem solving within immersive virtual worlds. Students often share information, ask questions, negotiate, and construct explanations between themselves towards solving a common goal. However, students sometimes disengage from the learning activities, and due to the nature of collaboration, their disengagement can propagate and negatively impact others within the group. From a teacher's perspective, it can be challenging to identify disengaged students within different groups in a classroom as they need to spend a significant amount of time orchestrating the classroom. Prior work has explored automated frameworks for identifying behavioral disengagement. However, most prior work relies on a single modality for identifying disengagement. In this work, we investigate the effects of using multiple modalities to detect disengagement behaviors of students in a collaborative game-based learning environment. For that, we utilized facial video recordings and group chat messages of 26 middle school students while they were interacting with Crystal Island: EcoJourneys, a game-based learning environment for ecosystem science. Our study shows that the predictive accuracy of a unimodal model heavily relies on the modality of the ground truth, whereas multimodal models surpass the unimodal models, trading resources for accuracy. Our findings can benefit future researchers in designing behavioral engagement detection frameworks for assisting teachers in using collaborative game-based learning within their classrooms.
Fahmid M. Fahid, Seung Y. Lee, Bradford W. Mott, Jessica Vandenberg, Halim Acosta, Thomas A. Brush, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
LAK3
2023 Is Elementary AI Education Possible?
abstract
As artificial intelligence (AI) technology becomes increasingly pervasive, it is critical that students recognize AI and how it can be used. There is little research exploring learning capabilities of elementary students and the pedagogical supports necessary to facilitate students' learning. PrimaryAI was created as a 3rd-5th grade AI curriculum that utilizes problem-based and immersive learning within an authentic life science context through four units that cover machine learning, computer vision, AI planning, and AI ethics. The curriculum was implemented by two upper elementary teachers during Spring 2022. Based on pre-test/post-test results, students were able to conceptualize AI concepts related to machine learning and computer vision. Results showed no significant differences based on gender. Teachers indicated the curriculum engaged students and provided teachers with sufficient scaffolding to teach the content in their classrooms. Recommendations for future implementations include greater alignment between the AI and life science concepts, alterations to the immersive problem-based learning environment, and enhanced connections to local animal populations.
Anne T. Ottenbreit-Leftwich, Krista D. Glazewski, Cindy E. Hmelo-Silver, Katie Jantaraweragul, Minji Jeon, Srijita Chakraburty, J. Adam Scribner, Seung Y. Lee, Bradford W. Mott, James C. Lester
SIGCSE (2)9
2023 Supporting Upper Elementary Students in Multidisciplinary Block-Based Narrative Programming
abstract
Digital storytelling, which combines traditional storytelling with digital tools, has seen growing popularity as a means of creating motivating problem-solving activities in K-12 education. Though an attractive potential solution to integrating language arts skills across topic areas such as computational thinking and science, better understanding of how to structure and support these activities is needed to increase adoption by teachers. Building on prior research on block-based programming for interactive storytelling, we present initial results from a study of 28 narrative programs created by upper elementary students that were collected in both classroom and extracurricular contexts. The narrative programs are evaluated across multiple dimensions to better understand the types of narrative programs being created by the students, characteristics of the students who created the narratives, and what types of support could most benefit the students in their narrative program construction. In addition to analyzing the student-created narrative programs, we also provide recommendations for promising system-generated and instructor-led supports.
Jessica Vandenberg, Anisha Gupta, Andy Smith, Rasha Elsayed, Kimkinyona Fox, Aleata Hubbard Cheuoua, James Minogue, Kevin M. Oliver, Cathy Ringstaff, Bradford W. Mott
SIGCSE (2)10
2023 Promoting AI Education for Rural Middle Grades Students with Digital Game Design
abstract
The demand is growing for a populace that is literate in Artificial Intelligence (AI); such literacy centers on enabling individuals to evaluate, collaborate with, and effectively use AI. Because the middle school years are a critical time for developing youths' perceptions and dispositions toward STEM, creating engaging AI learning experiences for middle grades students (ages 11 to 14) is paramount. The need for providing enhanced access to AI learning opportunities is especially pronounced in rural areas, which are typically underserved and underresourced. Inspired by prior research that game design holds significant potential for cultivating student interest and knowledge in computer science, we are designing, developing, and iteratively refining an AI-centered development environment that infuses AI learning into game design activities. In this work, we review design principles for game design interventions focused on middle grades computer science education and explore how to introduce AI learning experiences into interactive game design activities. We also discuss results from our initial co-design sessions with middle grades students and teachers in rural communities.
Jessica Vandenberg, Wookhee Min, Veronica Cateté, Danielle Boulden, Bradford W. Mott
SIGCSE (2)5
2022 Investigating Student Interest and Engagement in Game-Based Learning Environments
Jiayi Zhang 0004, Stephen Hutt, Jaclyn Ocumpaugh, Nathan L. Henderson, Alex Goslen, Jonathan P. Rowe, Kristy Elizabeth Boyer, Eric N. Wiebe, Bradford W. Mott, James C. Lester
AIED (1)9
2022 Enhancing Stealth Assessment in Game-Based Learning Environments with Generative Zero-Shot Learning
Nathan L. Henderson, Halim Acosta, Wookhee Min, Bradford W. Mott, Trudi Lord, Frieda Reichsman, Chad Dorsey, Eric N. Wiebe, James C. Lester
EDM4
2022 PrimaryAI: Co-Designing Immersive Problem-Based Learning for Upper Elementary Student Learning of AI Concepts and Practices
abstract
There is growing awareness of the central role that artificial intelligence (AI) plays now and in children's futures. This has led to increasing interest in engaging K-12 students in AI education to promote their understanding of AI concepts and practices. Leveraging principles from problem-based pedagogies and game-based learning, our approach integrates AI education into a set of unplugged activities and a game-based learning environment. In this work, we describe outcomes from our efforts to co design problem-based AI curriculum with elementary school teachers.
Krista D. Glazewski, Anne T. Ottenbreit-Leftwich, Katie Jantaraweragul, Minji Jeon, Cindy E. Hmelo-Silver, J. Adam Scribner, Seung Y. Lee, Bradford W. Mott, James C. Lester
ITiCSE (2)8
2022 Principles for AI Education for Elementary Grades Students
abstract
AI is beginning to transform every aspect of society. With the dramatic increases in AI, K-12 students need to be prepared to understand AI. To succeed as the workers, creators, and innovators of the future, students must be introduced to core concepts of AI as early as elementary school. However, building a curriculum that introduces AI content to K-12 students present significant challenges, such as connecting to prior knowledge, and developing curricula that are meaningful for students and possible for teachers to teach. To lay the groundwork for elementary AI education, we conducted a qualitative study into the design of AI curricular approaches with elementary teachers and students. Interviews with elementary teachers and students suggests four design principles for creating an effective elementary AI curriculum to promote uptake by teachers. This example will present the co-designed curriculum with teachers (PRIMARYAI) and describe how these four elements were incorporated into real-world problem-based learning scenarios.
Anne T. Ottenbreit-Leftwich, Krista D. Glazewski, Minji Jeon, Katie Jantaraweragul, Cindy E. Hmelo-Silver, J. Adam Scribner, Seung Y. Lee, Bradford W. Mott, James C. Lester
ITiCSE (2)8
2022 Disruptive Talk Detection in Multi-Party Dialogue within Collaborative Learning Environments with a Regularized User-Aware Network
abstract
Kyungjin Park, Hyunwoo Sohn, Wookhee Min, Bradford Mott, Krista Glazewski, Cindy E. Hmelo-Silver, James Lester. Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue. 2022.
Kyungjin Park, Hyunwoo Sohn, Wookhee Min, Bradford W. Mott, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
SIGDIAL4
2021 AI-Infused Collaborative Inquiry in Upper Elementary School: A Game-Based Learning Approach
abstract
Artificial intelligence has emerged as a technology that is profoundly reshaping society and enabling rapid improvements in science, engineering, and mathematics, as well as information technology itself. This has generated increased demand for fostering an AI-literate populace as well as a growing recognition of the importance of promoting K-12 students’ awareness and interest in AI. Although efforts are be-ginning to incorporate AI learning within K-12 education, there is little research exploring how to introduce students to AI and how to support teachers to integrate AI learning experiences in their classrooms. This is especially true at the elementary school level. A particularly promising approach for providing effective and engaging AI learning experiences for elementary students is game-based learning. In this paper, we explore how to introduce AI-infused collaborative inquiry learning into upper elementary school (student ages 8 to 11) using game-based learning. To ground the work in the realities of elementary school classrooms, we present insights from interviews with elementary school teachers to under-stand how best to support them in integrating AI into their classrooms. We then present the design of PrimaryAI, a game-based learning environment that supports rich problem-based learning activities within upper elementary classrooms centered on AI applied toward solving life-science problems. Finally, we discuss some of the challenges we face in bringing AI-infused collaborative inquiry learning to upper elementary students.
Seung Y. Lee, Bradford W. Mott, Anne T. Ottenbreit-Leftwich, J. Adam Scribner, Sandra Taylor, Kyungjin Park, Jonathan P. Rowe, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
AAAI2
2021 Multidimensional Team Communication Modeling for Adaptive Team Training: A Hybrid Deep Learning and Graphical Modeling Framework
Wookhee Min, Randall Spain, Jason D. Saville, Bradford W. Mott, Keith W. Brawner, Joan Hall Johnston, James C. Lester
AIED (1)4
2021 Modeling Frustration Trajectories and Problem-Solving Behaviors in Adaptive Learning Environments for Introductory Computer Science
Xiaoyi Tian 0001, Joseph B. Wiggins, Fahmid M. Fahid, Andrew Emerson, Dolly Bounajim, Andy Smith, Kristy Elizabeth Boyer, Eric N. Wiebe, Bradford W. Mott, James C. Lester
AIED (2)9
2021 "What's Important to You, Max?": The Influence of Goals on Engagement in an Interactive Narrative for Adolescent Health Behavior Change
Megan Mott, Bradford W. Mott, Jonathan P. Rowe, Elizabeth Ozer, Alison Giovanelli, Mark Berna, Marianne Pugatch, Kathleen Tebb, Carlos Penilla, James C. Lester
ICIDS2
2021 Supporting Interactive Storytelling with Block-Based Narrative Programming
Andy Smith, Danielle Boulden, Bradford W. Mott, Aleata Hubbard Cheuoua, James Minogue, Kevin M. Oliver, Cathy Ringstaff
ICIDS3
2021 Supporting Students' Computer Science Learning with a Game-based Learning Environment that Integrates a Use-Modify-Create Scaffolding Framework
abstract
Use-Modify-Create (UMC) has gained recognition as a viable scaffolding approach for student programming activities, but little is known about how UMC could support CS learning in game-based learning environments. We designed and developed a game to teach middle grade students (ages 11-13) CS through block-based programming challenges. The game integrates a UMC pedagogical framework to promote successful student outcomes for a wide variety of student abilities, including those without prior programming experience. Utilizing a mixed-methods research design, we investigated how the game influenced student learning of CS concepts and the role of UMC on the problem-solving strategies students applied to complete the game. In particular, we were interested in how prior experience would moderate these outcomes. Results from a multilevel model of students' pre-and post-assessment scores (N = 77) on a CS concepts assessment indicated that all students, regardless of prior programming experience, showed significant learning gains from pre to post after playing the game. Qualitative results revealed that the UMC scaffolding progression provided students, particularly those with little to no prior programming experience, with the foundational knowledge needed to progress through the game levels and challenges. Specifically, we found that the Use phases of the game reduced novice students' cognitive load and facilitated the necessary CS conceptual understanding to solve the open-ended programming tasks encountered in the game's Modify and Create phases. Our findings demonstrate the efficacy of UMC to support the learning of novice programmers in a game-based learning environment while not to the detriment of those more experienced.
Danielle Boulden, Arif Rachmatullah, Madeline Hinckle, Dolly Bounajim, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester, Eric N. Wiebe
ITiCSE (1)5
2021 Promoting Computational Thinking in Elementary School: A Narrative-Centered Learning Approach
abstract
One of the most efficient ways for elementary school students to gain exposure to computational thinking is when it is integrated into other disciplinary areas; however, elementary school teachers often lack the necessary resources to do this effectively. By leveraging the motivation force of narrative to engage students and the scaffolding affordances of block-based programming to support students, computationally-rich narrative-centered learning offers promise to address this need. In this work, we review design principles from prior work for engaging elementary students in computational thinking as well as results from initial pilot studies to investigate how computationally-rich narrative-centered learning in the context of science problem solving can support the integration of computational thinking into other disciplinary areas.
Danielle Boulden, Andy Smith, Kimkinyona Fox, Jennifer Houchins, Rasha Elsayed, Aleata Hubbard Cheuoua, James Minogue, Kevin M. Oliver, Cathy Ringstaff, Bradford W. Mott
ITiCSE (2)10
2021 Detecting Disruptive Talk in Student Chat-Based Discussion within Collaborative Game-Based Learning Environments
abstract
Collaborative game-based learning environments offer significant promise for creating engaging group learning experiences. Online chat plays a pivotal role in these environments by providing students with a means to freely communicate during problem solving. These chat-based discussions and negotiations support the coordination of students’ in-game learning activities. However, this freedom of expression comes with the possibility that some students might engage in undesirable communicative behavior. A key challenge posed by collaborative game-based learning environments is how to reliably detect disruptive talk that purposefully disrupt team dynamics and problem-solving interactions. Detecting disruptive talk during collaborative game-based learning is particularly important because if it is allowed to persist, it can generate frustration and significantly impede the learning process for students. This paper analyzes disruptive talk in a collaborative game-based learning environment for middle school science education to investigate how such behaviors influence students’ learning outcomes and varies across gender and students’ prior knowledge. We present a disruptive talk detection framework that automatically detects disruptive talk in chat-based group conversations. We further investigate both classic machine learning and deep learning models for the framework utilizing a range of dialogue representations as well as supplementary information such as student gender. Findings show that long short-term memory network (LSTM)-based disruptive talk detection models outperform competitive baseline models, indicating that the LSTM-based disruptive talk detection framework offers significant potential for supporting effective collaborative game-based learning through the identification of disruptive talk.
Kyungjin Park, Hyunwoo Sohn, Bradford W. Mott, Wookhee Min, Asmalina Saleh, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
LAK3
2021 How do Elementary Students Conceptualize Artificial Intelligence?
abstract
Countries around the globe have acknowledged the pervasiveness of artificial intelligence (AI) in our lives and the importance of educating our students on how AI technologies work. For example, the Chinese education ministry has integrated AI into the mandatory high school curriculum, including a pilot textbook to teach students about the fundamental AI technologies like deep learning and recognition. However, there are fewer examples of AI education at the primary level, and these are typically focused on decision-making, machine learning, and programming. We have recently started to develop our own elementary AI curriculum. To develop the curriculum, we first needed to explore what students already knew about AI. Although there are a few small studies on how primary students conceptualize AI, we conducted a study to examine how 10 nine- and ten-year-old students conceptualized artificial intelligence by focusing on two main questions: (1) How do elementary students conceptualize artificial intelligence? (2) What are elementary students' experiences with artificial intelligence? Students? definitions of AI tended to focus on programming and robotics. Students described examples of AI that included robotic vacuums, Siri/Alexa, YouTube, and search engines. They also showcased some misconceptions around AI designs and implementations.
Anne T. Ottenbreit-Leftwich, Krista D. Glazewski, Minji Jeon, Cindy E. Hmelo-Silver, Bradford W. Mott, Seung Y. Lee, James C. Lester
SIGCSE5
2021 Exploring Novice Programmers' Hint Requests in an Intelligent Block-Based Coding Environment
abstract
Block-based programming environments are widely used by novices who are learning computer science. However, even in block-based coding environments that have been carefully developed to serve novices, students frequently struggle and require additional support. A promising avenue to provide this support is the use of intelligent tutoring systems, which offer adaptive hints to assist learners. In order to provide students with the adaptive hints they need, we must investigate their help-seeking behaviors and identify patterns surrounding their need for support. In this experience report, we examine data collected from 174 college students in an introductory engineering course, who used an intelligent block-based coding environment to learn computer science. These students made more than 1,000 hint requests, which we represent in two-dimensional space along axes of elapsed time and code completeness. Analysis revealed five major clusters of hint requests, which we further characterized through qualitative examination of the coding trajectories that preceded each hint request. We also analyzed how students' incoming knowledge and perceived computer skill were related to their help-seeking behaviors. Students with higher incoming knowledge requested hints when their code was more complete than students with lower incoming knowledge. Students with high perceived computer skill asked for hints when their code was less complete than those with low perceived computer skill. The results presented here provide insight into student help-seeking behavior in computer science education, informing CS educators and system designers on how best to develop support strategies.
Joseph B. Wiggins, Fahmid M. Fahid, Andrew Emerson, Madeline Hinckle, Andy Smith, Kristy Elizabeth Boyer, Bradford W. Mott, Eric N. Wiebe, James C. Lester
SIGCSE7
2021 Progression Trajectory-Based Student Modeling for Novice Block-Based Programming
abstract
Block-based programming environments are widely used in computer science education. However, these environments pose significant challenges for student modeling. Given a series of problem-solving actions taken by students in block-based programming environments, student models need to accurately infer problem-solving students’ programming abilities in real time to enable adaptive feedback and hints that are tailored to students’ abilities. While student models for block-based programming offer the potential to support student-adaptivity, creating student models for these environments is challenging because students can develop a broad range of solutions to a given programming activity. To address these challenges, we introduce a progression trajectory-based student modeling framework for modeling novice student block-based programming across multiple learning activities. Student trajectories utilize a time series representation that employs code analysis to incrementally compare student programs to expert solutions as students undertake block-based programming activities. This paper reports on a study in which progression trajectories were collected from more than 100 undergraduate students engaging in a series of block-based programming activities in an introductory computer science course. Using progression trajectory-based student modeling, we identified three distinct trajectory classes: Early Quitting, High Persistence, and Efficient Completion. Analysis of these trajectories revealed that they exhibit significantly different characteristics with respect to students’ actions and can be used to accurately predict students’ programming behaviors on future programming activities compared to competing baseline models. The findings suggest that progression trajectory-based student models can accurately model students’ block-based programming problem solving and hold potential for informing adaptive support in block-based programming environments.
Fahmid M. Fahid, Xiaoyi Tian 0001, Andrew Emerson, Joseph B. Wiggins, Dolly Bounajim, Andy Smith, Eric N. Wiebe, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
UMAP8
2021 Designing a Visual Interface for Elementary Students to Formulate AI Planning Tasks
abstract
Recent years have seen the rapid adoption of artificial intelligence (AI) in every facet of society. The ubiquity of AI has led to an increasing demand to integrate AI learning experiences into K-12 education. Early learning experiences incorporating AI concepts and practices are critical for students to better understand, evaluate, and utilize AI technologies. AI planning is an important class of AI technologies in which an AI-driven agent utilizes the structure of a problem to construct plans of actions to perform a task. Although a growing number of efforts have explored promoting AI education for K-12 learners, limited work has investigated effective and engaging approaches for delivering AI learning experiences to elementary students. In this paper, we propose a visual interface to enable upper elementary students (grades 3–5, ages 8–11) to formulate AI planning tasks within a game-based learning environment. We present our approach to designing the visual interface as well as how the AI planning tasks are embedded within narrative-centered gameplay structured around a Use-Modify-Create scaffolding progression. Further, we present results from a qualitative study of upper elementary students using the visual interface. We discuss how the Use-Modify-Create approach supported student learning as well as discuss the misconceptions and usability issues students encountered while using the visual interface to formulate AI planning tasks.
Kyungjin Park, Bradford W. Mott, Seung Y. Lee, Krista D. Glazewski, J. Adam Scribner, Anne T. Ottenbreit-Leftwich, Cindy E. Hmelo-Silver, James C. Lester
VL/HCC2
2020 Detecting Off-Task Behavior from Student Dialogue in Game-Based Collaborative Learning
Dan Carpenter, Andrew Emerson, Bradford W. Mott, Asmalina Saleh, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
AIED (1)3
2020 Generating Game Levels to Develop Computer Science Competencies in Game-Based Learning Environments
Kyungjin Park, Bradford W. Mott, Wookhee Min, Eric N. Wiebe, Kristy Elizabeth Boyer, James C. Lester
AIED (2)2
2020 Promoting Computer Science Learning with Block-Based Programming and Narrative-Centered Gameplay
abstract
Recent years have seen increasing awareness of the need for all students in primary and secondary education to learn computer science (CS) concepts and skills. Educational games hold significant potential to serve as a platform for CS education because they integrate engaging problem solving with effective pedagogical strategies. This potential is especially high for narrative-centered educational games that embed learning activities within rich interactive stories. In this paper, we present an educational game featuring block-based programming challenges contextualized within an engaging narrative, designed to promote CS learning for middle school students (ages 11 to 13). In the game, students undertake problem-solving challenges that are aligned with the K-12 Computer Science Framework. Results from a classroom implementation of the game with middle grade students suggest that their perceived game control ratings are positively correlated with their progress in the game, which suggests the need for adaptively supporting students' game-based learning activities. Building on these findings, we discuss design implications for creating student-adaptive CS learning experiences in educational games that incorporate block-based programming enriched narrative-centered gameplay.
Wookhee Min, Bradford W. Mott, Kyungjin Park, Sandra Taylor, Bita Akram, Eric N. Wiebe, Kristy Elizabeth Boyer, James C. Lester
CoG2
2020 Automated Assessment of Computer Science Competencies from Student Programs with Gaussian Process Regression
Bita Akram, Hamoon Azizsoltani, Wookhee Min, Eric N. Wiebe, Bradford W. Mott, Anam Navied, Kristy Elizabeth Boyer, James C. Lester
EDM5
2020 Enhancing Student Competency Models for Game-Based Learning with a Hybrid Stealth Assessment Framework
Nathan L. Henderson, Vikram Kumara, Wookhee Min, Bradford W. Mott, Danielle Boulden, Trudi Lord, Frieda Reichsman, Chad Dorsey, Eric N. Wiebe, James C. Lester
EDM4
2020 Toward a Block-Based Programming Approach to Interactive Storytelling for Upper Elementary Students
Andy Smith, Bradford W. Mott, Sandra Taylor, Aleata Hubbard Cheuoua, James Minogue, Kevin M. Oliver, Cathy Ringstaff
ICIDS2
2020 The Relationship of Gender, Experiential, and Psychological Factors to Achievement in Computer Science
abstract
Computer science (CS) is widely recognized as a field with a significant gender gap despite the growing prevalence of computing. Several factors including CS attitudes, exposure to CS, experience with computer programming, and confidence in using computers are understood to be correlated with the low participation of women in CS. These factors also play an important role in students' interest in CS careers and are particularly crucial during secondary school. However, there is a dearth of research that examines differences in how these factors are inter-correlated for younger students (ages 11-13). The purpose of this study was to generate and test a statistical model that demonstrates the inter-correlation amongst these factors with respect to gender. A total of 260 middle school students participated in this study. Four instruments measuring students' CS attitudes, confidence in using computers, CS conceptual understanding, and prior experience with CS-related activities were used. Structural equation modeling was utilized to test the hypothesized model. The findings showed that previous participation in CS-related activities had a significant direct effect on CS attitudes and confidence in using computers, but the effect on students' CS conceptual understanding was indirect. We also found that in a female specific model, previous participation had a significantly stronger direct effect on CS attitudes compared to its effect in a male specific model. The importance of providing more CS-related experience, especially to female students, as well as suggestions on activities that promote gender equity in the field are discussed.
Madeline Hinckle, Arif Rachmatullah, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester, Eric N. Wiebe
ITiCSE3
2020 Designing a Collaborative Game-Based Learning Environment for AI-Infused Inquiry Learning in Elementary School Classrooms
abstract
Recent years have seen growing recognition of the importance of enabling K-12 students to learn computer science. Meanwhile, artificial intelligence has emerged as a technology with the potential to profoundly reshape society. This has generated increasing demand for fostering an AI-literate populace. However, there is little work exploring how to introduce K-12 students to AI and how to support K-12 teachers in integrating AI into their classrooms. In this work, we explore introducing AI learning experiences into upper elementary classrooms (student ages 8 to 11). With a focus on integrating AI and life science, we present initial work on a collaborative game-based learning environment that features rich problem-based learning scenarios. This will enable students to gain experience with AI as it applies to solving real-world life-science problems.
Seung Y. Lee, Bradford W. Mott, Anne T. Ottenbreit-Leftwich, J. Adam Scribner, Sandra Taylor, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
ITiCSE2
2020 A Conceptual Assessment Framework for K-12 Computer Science Rubric Design
abstract
The lack of effective guidelines for assessing students' computer science (CS) competencies is creating significant demand by K-12 teachers for CS assessments to evaluate students' learning. We propose a conceptual assessment framework that guides teachers through designing appropriate assessments for computer science (CS) activities in their classrooms. The framework addresses the critical problem of incorporating CS into K-12 curricula without corresponding assessments. We illustrate its use with the design of a rubric for a bubble sort algorithm situated in a game-based learning environment for middle-grade students. We also apply a preliminary and a revised version of this assessment on two datasets collected from students' interactions with the learning environment. We found consistency among results identified through applying the preliminary and the revised rubric. The results reveal distinctive patterns in students' approaches to CS problem solving and coherency with respect to different aspects of the rubric.*
Bita Akram, Wookhee Min, Eric N. Wiebe, Anam Navied, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
SIGCSE5
2020 Exploring Middle School Students' Reflections on the Infusion of CS into Science Classrooms
abstract
In recent years, there has been a dramatic increase in teaching CS in the context of other disciplines such as science. However, learning CS in an interdisciplinary context may be particularly challenging for students. An important goal for CS education researchers is to develop a deep understanding of the student experience when integrating CS into science classrooms in K-12. This paper presents the results of a mixed-methods study in which 75 middle school students engaged in a series of computationally rich science activities by creating simulations and models in a block-based programming language. After two semesters, students reported their experiences on in-class computer science activities through reflection essays. The quantitative results show that both experienced and novice students increased their CS knowledge significantly after several weeks, and a majority of students (72%) had positive sentiment toward the integration of CS into their science class. Deeper qualitative analysis of students' reflections revealed positive themes centered around the visualization and gamification of science concepts, the hands-on nature of the coding activities, and showing science from a different angle. On the other hand, students expressed negative sentiments on weaknesses in the activity design, lack of CS/science background/interest, and failing to make connections between CS and science concepts. These findings inform efforts to infuse CS education into different disciplines and reveal patterns that may foster success of K-12 classroom implementations.
Mehmet Celepkolu, David Austin Fussell, Aisha Chung Galdo, Kristy Elizabeth Boyer, Eric N. Wiebe, Bradford W. Mott, James C. Lester
SIGCSE6
2020 Cluster-Based Analysis of Novice Coding Misconceptions in Block-Based Programming
abstract
Recent years have seen an increasing interest in identifying common student misconceptions during introductory programming. In a parallel development, block-based programming environments for novice programmers have grown in popularity, especially in introductory courses. While these environments eliminate many syntax-related errors faced by novice programmers, there has been limited work that investigates the types of misconceptions students might exhibit in these environments. Developing a better understanding of these misconceptions will enable these programming environments and instructors to more effectively tailor feedback to students, such as prompts and hints, when they face challenges. In this paper, we present results from a cluster analysis of student programs from interactions with programming activities in a block-based programming environment for introductory computer science education. Using the interaction data from students' programming activities, we identify three families of student misconceptions and discuss their implications for refinement of the activities as well as design of future activities. We then examine the value of block counts, block sequence counts, and system interaction counts as programming features for clustering block-based programs. These clusters can help researchers identify which students would benefit from feedback or interventions and what kind of feedback provides the most benefit to that particular student.
Andrew Emerson, Andy Smith, Fernando J. Rodríguez, Eric N. Wiebe, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
SIGCSE5
2020 Designing Block-Based Programming Language Features to Support Upper Elementary Students in Creating Interactive Science Narratives
abstract
Recent years have seen a growing recognition of the importance of enabling K-12 students to engage in computational thinking, particularly in elementary grades where students' dispositions toward STEM are developing. Block-based programming has emerged as an effective tool for engaging these novice learners in computational thinking. At the same time, digital storytelling has emerged as a promising avenue for creating motivating problem-solving scenarios that engage students in science investigations. Although block-based programming and digital storytelling are in many ways synergistic, there is a lingering question of how to design block-based languages at an age-appropriate level to enable effective and engaging storytelling. In this work, we review design principles from prior block-based and digital storytelling systems as well as propose the design of block-based programming language features to enable the creation of rich, interactive science narratives by upper elementary students.
Andy Smith, Bradford W. Mott, Sandra Taylor, Aleata Hubbard Cheuoua, James Minogue, Kevin M. Oliver, Cathy Ringstaff
SIGCSE2
2020 Predictive Student Modeling in Block-Based Programming Environments with Bayesian Hierarchical Models
abstract
Recent years have seen a growing interest in block-based programming environments for computer science education. Although block-based programming offers a gentle introduction to coding for novice programmers, introductory computer science still presents significant challenges, so there is a great need for block-based programming environments to provide students with adaptive support. Predictive student modeling holds significant potential for adaptive support in block-based programming environments because it can identify early on when a student is struggling. However, predictive student models often make a number of simplifying assumptions, such as assuming a normal response distribution or homogeneous student characteristics, which can limit the predictive performance of models. These assumptions, when invalid, can significantly reduce the predictive accuracy of student models.
Andrew Emerson, Michael Geden, Andy Smith, Eric N. Wiebe, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
UMAP5
2019 4D Affect Detection: Improving Frustration Detection in Game-Based Learning with Posture-Based Temporal Data Fusion
Nathan L. Henderson, Jonathan P. Rowe, Bradford W. Mott, Keith W. Brawner, Ryan Baker 0001, James C. Lester
AIED (1)3
2019 Predicting Dialogue Breakdown in Conversational Pedagogical Agents with Multimodal LSTMs
Wookhee Min, Kyungjin Park, Joseph B. Wiggins, Bradford W. Mott, Eric N. Wiebe, Kristy Elizabeth Boyer, James C. Lester
AIED (2)4
2019 Take the Initiative: Mixed Initiative Dialogue Policies for Pedagogical Agents in Game-Based Learning Environments
Joseph B. Wiggins, Mayank Kulkarni, Wookhee Min, Kristy Elizabeth Boyer, Bradford W. Mott, Eric N. Wiebe, James C. Lester
AIED (2)5
2019 Generating Educational Game Levels with Multistep Deep Convolutional Generative Adversarial Networks
abstract
Educational games offer significant potential for supporting personalized learning in engaging virtual worlds. However, many educational games do not provide adaptive gameplay to meet the needs of individual students. To address this issue, educational games should include game levels that can self-adjust to the specific needs of individual students. However, creating a large number of adaptable game levels requires considerable effort by game developers. A promising solution to this problem is to leverage procedural content generation to automatically generate levels for educational games that incorporate the desired learning objectives. In this paper, we propose a multistep deep convolutional generative adversarial network for generating new levels within a game for middle school computer science education. The model operates in two phases: (1) train a generator with a small set of human-authored example levels and generate a much larger set of synthetic levels to augment the training data for a second generator, and (2) train a second generator using the augmented training data and use it to generate novel educational game levels with enhanced solvability. We evaluate the performance of the model by comparing the novelty and solvability of generated levels between the two generators. Results suggest that the proposed multistep model significantly enhances the solvability of the generated levels with only minor degradation in the novelty of the generated levels.
Kyungjin Park, Bradford W. Mott, Wookhee Min, Kristy Elizabeth Boyer, Eric N. Wiebe, James C. Lester
CoG2
2019 Predicting Early and Often: Predictive Student Modeling for Block-Based Programming Environments
Andrew Emerson, Andy Smith, Cody Smith, Fernando J. Rodríguez, Wookhee Min, Eric N. Wiebe, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
EDM7
2019 Designing and Developing Interactive Narratives for Collaborative Problem-Based Learning
Bradford W. Mott, Robert G. Taylor, Seung Y. Lee, Jonathan P. Rowe, Asmalina Saleh, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
ICIDS1
2019 Assessing Middle School Students' Computational Thinking Through Programming Trajectory Analysis
abstract
With national K-12 education initiatives such as "CSForAll," block-based programming environments have emerged as widely used tools for teaching novice programming. A key challenge presented by block-based programming environments is assessing students' computational thinking (CT) and programming competencies. Developing assessment methods that can evaluate students' use of CT practices such as testing and refining, and developing and using appropriate algorithms, can help teachers evaluate students learning and provide appropriate scaffolding. In this work, we utilize an evidence-centered assessment design approach to devise a three-dimensional assessment to evaluate students' CT competencies based on evidence extracted from their programming trajectories in a block-based programming environment. In this assessment, the first dimension assesses students' knowledge of essential CT concepts, the second dimension assesses students' dynamic testing and refining strategies, and the third dimension assesses their overall problem-solving efficiency. We apply the assessment framework to data collected from students' interactions with a game-based learning environment designed to develop middle-grade students' CT competencies and programming skills. The results demonstrate that students' knowledge of basic CT constructs, such as appropriate use and combination of control structures, serves as the foundation for designing and implementing effective algorithms. Further, we assessed students testing and refining strategies over the three dimensions of novelty, positivity, and scale. The results demonstrate that students with higher algorithmic capabilities tend to make more novel, positive, and small-scale changes. The results reveal distinctive patterns in students' approaches to computational thinking problem solving and make a step toward identifying and assessing productive computational thinking practices.
Bita Akram, Wookhee Min, Eric N. Wiebe, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
SIGCSE4
2019 Development of a Lean Computational Thinking Abilities Assessment for Middle Grades Students
abstract
The recognition of middle grades as a critical juncture in CS education has led to the widespread development of CS curricula and integration efforts. The goal of many of these interventions is to develop a set of underlying abilities that has been termed computational thinking (CT). This goal presents a key challenge for assessing student learning: we must identify assessment items associated with an emergent understanding of key cognitive abilities underlying CT that avoid specialized knowledge of specific programming languages. In this work we explore the psychometric properties of assessment items appropriate for use with middle grades (US grades 6-8; ages 11-13) students. We also investigate whether these items measure a single ability dimension. Finally, we strive to recommend a "lean" set of items that can be completed in a single 50-minute class period and have high face validity. The paper makes the following contributions: 1) adds to the literature related to the emerging construct of CT, and its relationship to the existing CTt and Bebras instruments, and 2) offers a research-based CT assessment instrument for use by both researchers and educators in the field.
Eric N. Wiebe, Jennifer E. London, Osman Aksit, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
SIGCSE4
2018 Improving Stealth Assessment in Game-based Learning with LSTM-based Analytics
Bita Akram, Wookhee Min, Eric N. Wiebe, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
EDM4
2018 High-Fidelity Simulated Players for Interactive Narrative Planning
abstract
Interactive narrative planning offers significant potential for creating adaptive gameplay experiences. While data-driven techniques have been devised that utilize player interaction data to induce policies for interactive narrative planners, they require enormously large gameplay datasets. A promising approach to addressing this challenge is creating simulated players whose behaviors closely approximate those of human players. In this paper, we propose a novel approach to generating high-fidelity simulated players based on deep recurrent highway networks and deep convolutional networks. Empirical results demonstrate that the proposed models significantly outperform the prior state-of-the-art in generating high-fidelity simulated player models that accurately imitate human players’ narrative interactions. Using the high-fidelity simulated player models, we show the advantage of more exploratory reinforcement learning methods for deriving generalizable narrative adaptation policies.
Jonathan P. Rowe, Wookhee Min, Bradford W. Mott, James C. Lester
IJCAI4
2018 Introducing the Computer Science Concept of Variables in Middle School Science Classrooms
abstract
The K-12 Computer Science Framework has established that students should be learning about the computer science concept of variables as early as middle school, although the field has not yet determined how this and other related concepts should be introduced. Secondary school computer science curricula such as Exploring CS and AP CS Principles often teach the concept of variables in the context of algebra, which most students have already encountered in their mathematics courses. However, when strategizing how to introduce the concept at the middle school level, we confront the reality that many middle schoolers have not yet learned algebra. With that challenge in mind, this position paper makes a case for introducing the concept of variables in the context of middle school science. In addition to an analysis of existing curricula, the paper includes discussion of a day-long pilot study and the consequent teacher feedback that further supports the approach. The CS For All initiative has increased interest in bringing computer science to middle school classrooms; this paper makes an argument for doing so in a way that can benefit students' learning of both computer science and core science content.
Philip Sheridan Buffum, Kimberly Michelle Ying, Xiaoxi Zheng, Kristy Elizabeth Boyer, Eric N. Wiebe, Bradford W. Mott, David C. Blackburn, James C. Lester
SIGCSE6
2017 Inducing Stealth Assessors from Game Interaction Data
Wookhee Min, Megan Hardy Frankosky, Bradford W. Mott, Eric N. Wiebe, Kristy Elizabeth Boyer, James C. Lester
AIED3
2017 Affect Dynamics in Military Trainees Using vMedic: From Engaged Concentration to Boredom to Confusion
Jaclyn Ocumpaugh, Juan Miguel L. Andres, Ryan Baker 0001, Jeanine DeFalco, Luc Paquette, Jonathan P. Rowe, Bradford W. Mott, James C. Lester, Vasiliki Georgoulas, Keith W. Brawner, Robert A. Sottilare
AIED7
2017 "Thanks Alisha, Keep in Touch": Gender Effects and Engagement with Virtual Learning Companions
Lydia Pezzullo, Joseph B. Wiggins, Megan Hardy Frankosky, Wookhee Min, Kristy Elizabeth Boyer, Bradford W. Mott, Eric N. Wiebe, James C. Lester
AIED6
2017 Interactive Narrative Personalization with Deep Reinforcement Learning
abstract
Data-driven techniques for interactive narrative generation are the subject of growing interest. Reinforcement learning (RL) offers significant potential for devising data-driven interactive narrative generators that tailor players’ story experiences by inducing policies from player interaction logs. A key open question in RL-based interactive narrative generation is how to model complex player interaction patterns to learn effective policies. In this paper we present a deep RL-based interactive narrative generation framework that leverages synthetic data produced by a bipartite simulated player model. Specifically, the framework involves training a set of Q-networks to control adaptable narrative event sequences with long short-term memory network-based simulated players. We investigate the deep RL framework’s performance with an educational interactive narrative, Crystal Island. Results suggest that the deep RL-based narrative generation framework yields effective personalized interactive narratives.
Jonathan P. Rowe, Wookhee Min, Bradford W. Mott, James C. Lester
IJCAI4
2016 Mining Sequences of Gameplay for Embedded Assessment in Collaborative Learning
Philip Sheridan Buffum, Megan Hardy Frankosky, Kristy Elizabeth Boyer, Eric N. Wiebe, Bradford W. Mott, James C. Lester
EDM5
2016 Predicting Dialogue Acts for Intelligent Virtual Agents with Multimodal Student Interaction Data
Wookhee Min, Joseph B. Wiggins, Lydia Pezzullo, Alexandria K. Vail, Kristy Elizabeth Boyer, Bradford W. Mott, Megan Hardy Frankosky, Eric N. Wiebe, James C. Lester
EDM6
2016 Decomposing Drama Management in Educational Interactive Narrative: A Modular Reinforcement Learning Approach
Jonathan P. Rowe, Bradford W. Mott, James C. Lester
ICIDS3
2016 Player Goal Recognition in Open-World Digital Games with Long Short-Term Memory Networks
Wookhee Min, Bradford W. Mott, Jonathan P. Rowe, Barry Liu, James C. Lester
IJCAI2
2016 Integrating Real-Time Drawing and Writing Diagnostic Models: An Evidence-Centered Design Framework for Multimodal Science Assessment
Andy Smith, Osman Aksit, Wookhee Min, Eric N. Wiebe, Bradford W. Mott, James C. Lester
ITS5
2016 Empowering All Students: Closing the CS Confidence Gap with an In-School Initiative for Middle School Students
abstract
The important goal of broadening participation in computing has inspired many successful outreach initiatives. Yet many of these initiatives, such as out-of-school activities or innovative new computer science courses for secondary school students, may disproportionately attract students who already have prior interest and experience in computing. How, then, do we engage the silent majority of students who do not self-select computer science? This paper examines this question in the context of ENGAGE, an in-school outreach initiative for middle school students. ENGAGE's learning activities center on a game-based learning environment for computer science. Results reveal that the initiative improved the computer science attitudes of students who were not already predisposed to study computer science, in a way that a corresponding after-school program could not. The results illustrate how an in-school initiative can empower young students who might not otherwise consider studying computer science.
Philip Sheridan Buffum, Megan Hardy Frankosky, Kristy Elizabeth Boyer, Eric N. Wiebe, Bradford W. Mott, James C. Lester
SIGCSE5
2015 Mind the Gap: Improving Gender Equity in Game-Based Learning Environments with Learning Companions
Philip Sheridan Buffum, Kristy Elizabeth Boyer, Eric N. Wiebe, Bradford W. Mott, James C. Lester
AIED4
2015 Modeling Self-Efficacy Across Age Groups with Automatically Tracked Facial Expression
Joseph F. Grafsgaard, Seung Y. Lee, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
AIED3
2015 Two Modes are Better Than One: a Multimodal Assessment Framework Integrating Student Writing and Drawing
Samuel P. Leeman-Munk, Andy Smith, Bradford W. Mott, Eric N. Wiebe, James C. Lester
AIED3
2015 DeepStealth: Leveraging Deep Learning Models for Stealth Assessment in Game-Based Learning Environments
Wookhee Min, Megan Hardy Frankosky, Bradford W. Mott, Jonathan P. Rowe, Eric N. Wiebe, Kristy Elizabeth Boyer, James C. Lester
AIED3
2015 Sensor-Free or Sensor-Full: A Comparison of Data Modalities in Multi-Channel Affect Detection
Luc Paquette, Jonathan P. Rowe, Ryan Baker 0001, Bradford W. Mott, James C. Lester, Jeanine DeFalco, Keith W. Brawner, Robert A. Sottilare, Vasiliki Georgoulas
EDM4
2015 ENGAGE: A Game-based Learning Environment for Middle School Computational Thinking
abstract
We present ENGAGE, a game-based learning environment for teaching computational thinking to middle school students. This project has dual aims: introducing computational thinking practices to students at a young age, and improving computational thinking attitudes among underrepresented students. In pursuit of these two goals, the ENGAGE team has mapped the learning objectives of the AP CS Principles course to the middle school level, and then built an immersive game experience upon that foundation. Students choose computer scientist avatars to represent themselves, and then play in pairs as they investigate a data-related mystery in an underwater research station, solving computational thinking challenges along the way. ENGAGE is currently being implemented as part of a quarterly elective in four middle schools in North Carolina. During the elective, students spend a total of ten classroom sessions playing the game, supplemented by "unplugged" activities that reinforce concepts learned in the game environment. We plan to expand to more middle schools in the 2015-2016 school year. In this demo, members of the SIGCSE community will be able to experience the ENGAGE game for themselves and learn more about its development and future directions. We will also discuss our success in recruiting and teaching the ENGAGE curriculum to middle school teachers who had no prior computer science experience, and the success of those middle school teachers in implementing ENGAGE within their classrooms.
Kristy Elizabeth Boyer, Philip Sheridan Buffum, Kirby Culbertson, Megan Hardy Frankosky, James C. Lester, Allison G. Martínez-Arocho, Wookhee Min, Bradford W. Mott, Fernando J. Rodríguez, Eric N. Wiebe
SIGCSE8
2015 Diagrammatic Student Models: Modeling Student Drawing Performance with Deep Learning
Andy Smith, Wookhee Min, Bradford W. Mott, James C. Lester
UMAP3
2014 SKETCHMINER: Mining Learner-Generated Science Drawings with Topological Abstraction
Andy Smith, Eric N. Wiebe, Bradford W. Mott, James C. Lester
EDM3
2014 FLARE: An open source toolkit for creating expressive user interfaces for serious games
Bradford W. Mott, Jonathan P. Rowe, Wookhee Min, Robert G. Taylor, James C. Lester
FDG1
2014 Play in the museum: Designing game-based learning environments for informal education settings
Jonathan P. Rowe, Eleni V. Lobene, Bradford W. Mott, James C. Lester
FDG3
2014 Leveraging Semi-Supervised Learning to Predict Student Problem-Solving Performance in Narrative-Centered Learning Environments
Wookhee Min, Bradford W. Mott, Jonathan P. Rowe, James C. Lester
Intelligent Tutoring Systems2
2014 Serious Games Go Informal: A Museum-Centric Perspective on Intelligent Game-Based Learning
Jonathan P. Rowe, Eleni V. Lobene, Bradford W. Mott, James C. Lester
Intelligent Tutoring Systems3
2014 Generalizability of Goal Recognition Models in Narrative-Centered Learning Environments
Alok Baikadi, Jonathan P. Rowe, Bradford W. Mott, James C. Lester
UMAP3
2014 Designing game-based learning environments for elementary science education: A narrative-centered learning perspective
James C. Lester, Hiller A. Spires, John L. Nietfeld, James Minogue, Bradford W. Mott, Eleni V. Lobene
Inf. Sci.5
2014 A Supervised Learning Framework for Modeling Director Agent Strategies in Educational Interactive Narrative
abstract
Computational models of interactive narrative offer significant potential for creating educational game experiences that are procedurally tailored to individual players and support learning. A key challenge posed by interactive narrative is devising effective director agent models that dynamically sequence story events according to players' actions and needs. In this paper, we describe a supervised machine-learning framework to model director agent strategies in an educational interactive narrative Crystal Island. Findings from two studies with human participants are reported. The first study utilized a Wizard-of-Oz paradigm where human “wizards” directed participants through Crystal Island's mystery storyline by dynamically controlling narrative events in the game environment. Interaction logs yielded training data for machine learning the conditional probabilities of a dynamic Bayesian network (DBN) model of the human wizards' directorial actions. Results indicate that the DBN model achieved significantly higher precision and recall than naive Bayes and bigram model techniques. In the second study, the DBN director agent model was incorporated into the runtime version of Crystal Island, and its impact on students' narrative-centered learning experiences was investigated. Results indicate that machine-learning director agent strategies from human demonstrations yield models that positively shape players' narrative-centered learning and problem-solving experiences.
Seung Y. Lee, Jonathan P. Rowe, Bradford W. Mott, James C. Lester
IEEE Trans. Comput. Intell. AI Games3
2013 Personalizing Embedded Assessment Sequences in Narrative-Centered Learning Environments: A Collaborative Filtering Approach
Wookhee Min, Jonathan P. Rowe, Bradford W. Mott, James C. Lester
AIED3
2013 Discovering Behavior Patterns of Self-Regulated Learners in an Inquiry-Based Learning Environment
Jennifer Sabourin, Bradford W. Mott, James C. Lester
AIED2
2013 Utilizing Dynamic Bayes Nets to Improve Early Prediction Models of Self-regulated Learning
Jennifer Sabourin, Bradford W. Mott, James C. Lester
UMAP2
2012 Goal Recognition with Markov Logic Networks for Player-Adaptive Games
abstract
Goal recognition in digital games involves inferring players’ goals from observed sequences of low-level player actions. Goal recognition models support player-adaptive digital games, which dynamically augment game events in response to player choices for a range of applications, including entertainment, training, and education. However, digital games pose significant challenges for goal recognition, such as exploratory actions and ill-defined goals. This paper presents a goal recognition framework based on Markov logic networks (MLNs). The model’s parameters are directly learned from a corpus that was collected from player interactions with a non-linear educational game. An empirical evaluation demonstrates that the MLN goal recognition framework accurately predicts players’ goals in a game environment with exploratory actions and ill-defined goals.
Eunyoung Ha, Jonathan P. Rowe, Bradford W. Mott, James C. Lester
AAAI3
2012 Early Prediction of Student Self-Regulation Strategies by Combining Multiple Models
Jennifer Sabourin, Bradford W. Mott, James C. Lester
EDM2
2012 Real-Time Narrative-Centered Tutorial Planning for Story-Based Learning
Seung Y. Lee, Bradford W. Mott, James C. Lester
ITS2
2012 Exploring Inquiry-Based Problem-Solving Strategies in Game-Based Learning Environments
Jennifer Sabourin, Jonathan P. Rowe, Bradford W. Mott, James C. Lester
ITS3
2012 Predicting Student Self-regulation Strategies in Game-Based Learning Environments
Jennifer Sabourin, Lucy R. Shores, Bradford W. Mott, James C. Lester
ITS3
2011 Modeling Learner Affect with Theoretically Grounded Dynamic Bayesian Networks
Jennifer Sabourin, Bradford W. Mott, James C. Lester
ACII (1)2
2011 Generalizing Models of Student Affect in Game-Based Learning Environments
Jennifer Sabourin, Bradford W. Mott, James C. Lester
ACII (2)2
2011 Modeling Narrative-Centered Tutorial Decision Making in Guided Discovery Learning
Seung Y. Lee, Bradford W. Mott, James C. Lester
AIED2
2011 When Off-Task is On-Task: The Affective Role of Off-Task Behavior in Narrative-Centered Learning Environments
Jennifer Sabourin, Jonathan P. Rowe, Bradford W. Mott, James C. Lester
AIED3
2011 Director Agent Intervention Strategies for Interactive Narrative Environments
Seung Y. Lee, Bradford W. Mott, James C. Lester
ICIDS2
2010 Individual differences in gameplay and learning: a narrative-centered learning perspective
abstract
Narrative-centered learning environments are an important class of educational games that situate learning within rich story contexts. The work presented in this paper investigates individual differences in gameplay and learning during student interactions with a narrative-centered learning environment, Crystal Island. Findings reveal striking differences between high- and low-achieving science students in problem-solving effectiveness, attention to particular gameplay elements, learning gains and engagement ratings. High-achieving science students tended to demonstrate greater problem-solving efficiency, reported higher levels of interest and presence in the narrative environment, and demonstrated an increased focus on information gathering and information organization gameplay activities. Lower-achieving microbiology students gravitated toward novel gameplay elements, such as conversations with non-player characters and the use of laboratory testing equipment. The findings have implications for the design of broadly effective gameplay activities for narrative-centered learning environments, as well as investigations of scaffolding techniques to promote effective problem solving, improved learning outcomes and sustained engagement for all students.
Jonathan P. Rowe, Lucy R. Shores, Bradford W. Mott, James C. Lester
FDG3
2010 Optimizing Story-Based Learning: An Investigation of Student Narrative Profiles
Seung Y. Lee, Bradford W. Mott, James C. Lester
Intelligent Tutoring Systems (2)2
2010 Integrating Learning and Engagement in Narrative-Centered Learning Environments
Jonathan P. Rowe, Lucy R. Shores, Bradford W. Mott, James C. Lester
Intelligent Tutoring Systems (2)3
2010 Exploring the Effectiveness of Lexical Ontologies for Modeling Temporal Relations with Markov Logic
Eunyoung Ha, Alok Baikadi, Carlyle Licata, Bradford W. Mott, James C. Lester
SIGDIAL Conference4
2008 Modeling self-efficacy in intelligent tutoring systems: An inductive approach
Scott W. McQuiggan, Bradford W. Mott, James C. Lester
User Model. User Adapt. Interact.2
2006 Probabilistic Goal Recognition in Interactive Narrative Environments
Bradford W. Mott, Sunyoung Lee, James C. Lester
AAAI1
2006 Narrative-Centered Tutorial Planning for Inquiry-Based Learning Environments
Bradford W. Mott, James C. Lester
Intelligent Tutoring Systems1
1999 Integrating discourse and domain knowledge for document drafting
Karl Branting, Charles B. Callaway, Bradford W. Mott, James C. Lester
ICAIL3
1999 An Integrated Scenario Management Strategy
abstract
Scenarios have proven effective for eliciting, describing and validating software requirements; however, scenario management continues to be a significant challenge to practitioners. One reason for this difficulty is that the number of possible relations among scenarios grows exponentially with the number of scenarios. If these relations are formalized, they can be more easily identified and supported. To provide this support, we extend the benefits of project-wide glossaries with two complementary approaches. The first approach employs shared scenario elements to identify and maintain common episodes among scenarios. The resulting episodes impose consistency across related scenarios and provide a way to visualize their interdependencies. The second approach quantifies similarity between scenarios. The resulting similarity measures serve as heuristics for finding duplicate scenarios, scenarios needing further elaboration, and scenarios which have not yet been identified yielding valuable information about how well the scenarios provide coverage of the requirements. These two approaches, integrated with a scenario database, project glossaries, configuration management, and coverage analysis, form the basis of a useful and effective strategy for scenario management and evolution.
Thomas A. Alspaugh, Annie I. Antón, Tiffany Barnes, Bradford W. Mott
RE4