Wookhee Min

dblp:131/2883 · DBLP profile ↗
← Back
62ranked-venue papers
9as first author
34since 2021 · last 2026
0000-0001-8900-0514ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Human-computer interaction and ubiquitous computing · 45 · 7 first-author · 25 since 2021Applied, interdisciplinary, general and emerging computing · 29 · 7 first-author · 14 since 2021Graphics, computer vision, multimedia, augmented reality and games · 15 · 2 first-author · 8 since 2021Artificial intelligence and machine learning · 9 · 1 first-author · 6 since 2021
YearPublicationVenuePosition
2026 An Explanation-Based Classroom Response System for Real-Time Analysis of Undergraduate Students' Natural Language Explanations
abstract
Effective classroom teaching requires instructors to be responsive to their students, such as by pivoting their lectures in real-time to address common misconceptions that their students may have developed. Classroom response systems such as multiple-choice "clicker" systems are one method by which instructors can gauge their students’ understanding during classroom lectures, but open-ended questions that prompt students to engage in self-explanation are better suited to promoting critical thinking. Additionally, analyzing students’ natural language responses typically requires time-consuming manual analysis, which makes it challenging to implement in a classroom setting. To address this challenge, we present an LLM-driven method for automatically assessing students' responses and generating an aggregated summary of LLM-based evaluations for their self-explanations during undergraduate classroom lectures. Our approach extracts relevant knowledge components for a given question, tags students’ responses according to whether they correctly address each knowledge component, and generates class-level summaries that highlight common misconceptions and gaps in knowledge to support instructors in pivoting their lectures in real time. We evaluate the system’s effectiveness at these tagging and summarization tasks on data from an undergraduate computer science course, using quantitative and qualitative metrics such as relevance, sufficiency, hallucination rate, and alignment with instructional goals and desired feedback format gathered through instructor interviews. Results suggest that the explanation-based classroom response system can accurately analyze students’ natural language explanations.
Jordan Esiason, Priyanka Khare, Claire Aguiar, Dan Carpenter, Wookhee Min, Seung Lee, Gamze Ozogul, James C. Lester
AAAI5
2026 A Dialogue-Based Learning Analytics Framework for Collaborative Game-Based Learning
abstract
In computer-supported collaborative learning environments, analyzing student dialogue is essential for understanding collaborative problem-solving behaviors and supporting effective learning. Prior work often treats all dialogue interactions uniformly, failing to capture how specific dialogue interaction differentially impact learning experiences and outcomes. To address this limitation, we introduce a dialogue-based learning analytics framework that integrates weighted temporal clustering of dialogue with large language model-based interpretation. Our framework identifies student interaction patterns most predictive of group learning gains and uses these insights to enable early prediction of learning outcomes and generate pedagogically meaningful interpretations. We evaluate our framework on collaborative dialogue from middle school students engaged in a collaborative game-based learning environment. Our results show that our framework achieves 83.1% accuracy in learning outcome prediction. In addition, expert evaluations and case studies demonstrate that the identified weighted dialogue patterns reflect key collaborative problem-solving behaviors recognized as important in collaborative learning. By surfacing high-impact interaction patterns and enabling prioritized interpretation generation, our framework provides a promising approach for accurately analyzing students’ collaborative dialogue.
Yeo Jin Kim, Daeun Hong, Wookhee Min, Snigdha Chaturvedi, Cindy E. Hmelo-Silver, James C. Lester
AAAI4
2026 From Embeddings to Chatbots: Playful NLP Activities for Middle School AI Literacy
abstract
As large language models (LLMs) and chatbots become increasingly prevalent, there is an urgent need to create engaging, age-appropriate learning activities that foster foundational AI literacy with a focus on natural language processing (NLP). This paper presents the iterative design and implementation of three instructional activities that introduce middle school learners (ages 11--14) to NLP concepts through playful, hands-on experiences aligned with the AI4K12 Big Idea of Natural Interaction. These activities include: (1) an unplugged card game that develops students' understanding of embeddings and similarity, (2) an unplugged collaborative sentence-generation challenge that illustrates how language models work, and (3) a web-based educational game in which students design and interact with chatbots. Each activity was implemented and refined across multiple educational contexts, including teacher professional development workshops, summer camps, and classroom implementations. All activities are designed to be easy to set up, requiring only commonly available classroom technology (e.g., laptops) and a few inexpensive materials (e.g., decks of cards), and are supported with facilitation guides and reflection prompts. Early implementations revealed areas for refinement, leading to clearer scaffolding that helped students connect gameplay to underlying NLP concepts, and post-refinement surveys indicated that students found the activities both enjoyable and educational. Findings suggest that blending unplugged and digital formats enhances comprehension, and that tailoring content to students' local contexts supports engagement. By making these activities openly available, this work contributes to the growing ecosystem of K–12 AI education resources and offers practical guidance for integrating NLP concepts into classroom instruction.
Jessica Vandenberg, Alex Goslen, Claire Aguiar, Wookhee Min, Veronica Cateté, Bradford W. Mott
AAAI4
2026 Multi-label Collaborative Dialogue Act Recognition for Adaptive Team Training Environments
Jay Pande, Wookhee Min, Randall Spain, Vikram Kumaran, James C. Lester
AIED (3)2
2026 Generating Clue-Driven Investigative Game Narratives with Large Language Models
Vikram Kumaran, Andy Smith, Wookhee Min, Randall Spain, Bradford W. Mott, James C. Lester
FDG3
2026 Play, Explore, Reward: Introducing Reinforcement Learning Concepts through Game-Based Learning in Middle School
Veronica Cateté, Deniz Ozturk, Claire Aguiar, Jessica Vandenberg, Wookhee Min, Bradford W. Mott
ITiCSE (1)5
2026 Collaborative Dialogue Analysis for Productive Problem Solving
abstract
Collaborative problem solving requires students to jointly reason, negotiate, and regulate their learning. Understanding collaborative problem solving through student dialogue can inform timely identification of productive and unproductive collaborative behaviors. In this study, we investigate the use of large language models to automatically classify collaborative problem-solving dialogue segments into two categories: Productive and Unproductive. To support deeper analysis, we additionally explore classification of eight detailed collaborative problem-solving sub-categories. We present an error-augmented few-shot prompting method that incorporates misclassified examples to refine model understanding of classification boundaries. Using dialogue data from a middle school collaborative game-based learning environment, our approach substantially improves classification accuracy over zero-shot baselines. Qualitative analysis of the resulting models further highlights which dialogue types are most frequently misclassified, suggesting design implications for adaptive scaffolding. These findings demonstrate that large language models, when guided with targeted prompting strategies, can effectively recognize productive and unproductive dialogue in collaborative learning.
Yeo Jin Kim, Daeun Hong, Xiaotian Zou, Cindy E. Hmelo-Silver, Wookhee Min, Snigdha Chaturvedi, James C. Lester
LAK5
2026 Topic-Level Feedback Summarization for an Explanation-Based Classroom Response System
abstract
Fostering engagement among undergraduate computer science students in large-lecture settings can be challenging for instructors. Didactic teaching styles common in such lectures may not be as effective as dialogic teaching, but the overhead involved with dialogic teaching may preclude its use in large introductory computer science courses. Classroom response systems such as multiple-choice ''clicker'' systems provide a way for students to engage with an instructor, but evidence suggests that multiple-choice questions may not foster deep thought in the way that open-ended questions do. Open-ended questions foster deeper engagement and AI-enabled learning analytics offer a powerful method of automatically assessing student responses, but grading text responses produced by students and summarizing class-wide performance during lectures presents unique difficulties, especially for algorithmic questions prevalent in computer science lectures.
Jordan Esiason, Wookhee Min, Seung Y. Lee, Gamze Ozogul, Yeil Jeong, James C. Lester
SIGCSE (2)2
2025 Shaping AI Interest in Rural Middle Schools with Unplugged Learning: Gender Differences and Teacher Insights
abstract
Adoption of artificial intelligence (AI) is at an inflection point. With daily use of AI escalating due to widely available software tools, educators, researchers, and policymakers must adapt swiftly to changing educational needs. While think tanks and Big Tech companies often promote the notion that AI serves as a powerful tool for democratizing access to knowledge and opportunities, our work in rural communities underscores the disparity in access to AI education and related opportunities. In this paper, we report on our experience introducing foundational AI concepts to rural middle school students using an unplugged game-based learning activity. By providing engaging learning experiences to rural populations, we hope to broaden interest in and understanding of AI technologies. To this end, we conducted a classroom study in which two middle school teachers implemented our unplugged AI learning activity with their students. Analyzing survey data from 60 of the participating students, we explore the impact of the activity on their interest in AI, their conceptual understanding, and examine potential gender differences. Additionally, we share insights from the teachers who participated in our professional development sessions in preparation for the classroom implementations.
Hansol Lim, Danielle Boulden, Jessica Vandenberg, Veronica Cateté, Wookhee Min, Bradford W. Mott
AAAI5
2025 Improving Student Modeling in Game-Based Learning with Multi-task Learning for Stealth Assessment and Goal Recognition
Anisha Gupta, Wookhee Min, Dan Carpenter, Roger Azevedo, James C. Lester
AIED (4)2
2025 Collaborative Problem-Solving Dialogue Analysis with Interpretable Temporal Clustering
Yeo Jin Kim, Daeun Hong, Wookhee Min, Snigdha Chaturvedi, Cindy E. Hmelo-Silver, James C. Lester
AIED (3)3
2025 Fostering AI Literacy Through Strategic Play: A Competitive Pathfinding Game for Middle School
Claire Aguiar, Dan Carpenter, Jessica Vandenberg, Wookhee Min, Veronica Cateté, Bradford W. Mott
CoG4
2025 A Multi-View Predictive Student Modeling Framework with Interpretable Causal Graph Discovery for Collaborative Learning Analytics
Halim Acosta, Seung Y. Lee, Daeun Hong, Wookhee Min, Bradford W. Mott, Cindy E. Hmelo-Silver, James C. Lester
EDM4
2025 A Multimodal Classroom Video Question-Answering Framework for Automated Understanding of Collaborative Learning
Nithin Sivakumaran, Chia-Yu Yang, Abhaysinh Zala, Shoubin Yu, Daeun Hong, Xiaotian Zou, Elias Stengel-Eskin, Dan Carpenter, Wookhee Min, Cindy E. Hmelo-Silver, Jonathan P. Rowe, James C. Lester, Mohit Bansal
ICMI9
2025 Collaborative Game-based Learning Analytics: Predicting Learning Outcomes from Game-based Collaborative Problem Solving Behaviors
Halim Acosta, Daeun Hong, Seung Y. Lee, Wookhee Min, Bradford W. Mott, Cindy E. Hmelo-Silver, James C. Lester
LAK4
2025 "Like a GPS": Analyzing Middle School Student Responses to an Interactive Pathfinding Activity
abstract
Engaging middle school students in complex computational topics such as AI can pose unique challenges to educators, ranging from simplifying potentially difficult mathematical material to maintaining student interest in the subject. One approach to help address these challenges is to utilize hands-on, real-world examples. We conducted a week-long summer camp for 24 students, centering each day around activities aligned with one of the Five Big Ideas in AI. Students participated in exit ticket reflections following each activity. The pathfinding activities, which occurred on one of the days, incorporated real-world examples and digital simulations of three pathfinding algorithms (breadth-first search, depth-first search, and A*), including an activity modeled after the game Pac-Man. Thematic analysis of exit-ticket responses revealed four major themes regarding students' key takeaways from the activities: (1) pathfinding for character movement, notably in video games like Minecraft; (2) pathfinding as a means of efficient navigation; (3) theoretical reasoning regarding the speed of the A* algorithm compared to others, highlighting its intelligent search mechanism; and (4) empirical reasoning based on personal experience during activities, where some students noted A* consistently performed fastest. These findings indicate that students not only engaged with AI concepts but also demonstrated a nuanced understanding of algorithmic efficiency. We examine the implications of these findings on understanding student engagement with interactive pathfinding activities and highlight the potential for future work in this area.
Claire Aguiar, Dan Carpenter, Jessica Vandenberg, Alex Goslen, Wookhee Min, Veronica Cateté, Bradford W. Mott
SIGCSE (2)5
2025 Introducing Reinforcement Learning Concepts to Middle School Students with Game-Based Learning
Matthew Presson, Anisha Gupta, Jessica Vandenberg, Alex Goslen, Wookhee Min, Veronica Cateté, Bradford W. Mott
SIGCSE (2)5
2024 Unplugged K-12 AI Learning: Exploring Representation and Reasoning with a Facial Recognition Game
abstract
With the growing prevalence of AI, the need for K-12 AI education is becoming more crucial, which is prompting active research in developing engaging and age-appropriate AI learning activities. Efforts are underway, such as those by the AI4K12 initiative, to establish guidelines for organizing K- 12 AI education; however, effective instructional resources are needed by educators. In this paper, we describe our work to design, develop, and implement an unplugged activity centered on facial recognition technology for middle school students. Facial recognition is integrated into a wide range of applications throughout daily life, which makes it a familiar and engaging tool for students and an effective medium for conveying AI concepts. Our unplugged activity, “Guess Whose Face,” is designed as a board game that focuses on Representation and Reasoning from AI4K12’s 5 Big Ideas in AI. The game is crafted to enable students to develop AI competencies naturally through physical interaction. In the game, one student uses tracing paper to extract facial features from a familiar face shown on a card, such as a cartoon character or celebrity, and then other students try to guess the identity of the hidden face. We discuss details of the game, its iterative refinement, and initial findings from piloting the activity during a summer camp for rural middle school students.
Hansol Lim, Wookhee Min, Jessica Vandenberg, Veronica Cateté, Bradford W. Mott
AAAI2
2024 Engaging Students from Rural Communities in AI Education with Game-Based Learning
abstract
As the presence of artificial intelligence (AI) technologies increases throughout everyday life, so does the need to engage rural communities in AI learning experiences, as these communities often have limited access to such educational opportunities. This work presents three game-based learning activities rooted in core AI concepts: natural language processing, search, and reinforcement learning. These activities were implemented in a summer camp with middle grades students in a rural area of the USA. We share an overview of the activities, as well as key observations and takeaways from student responses in post-activity surveys.
Alex Goslen, Anisha Gupta, Smrithi Muthukrishnan, Raven Midgett, Wookhee Min, Jessica Vandenberg, Veronica Cateté, Bradford W. Mott
SIGCSE (2)5
2024 Supporting Student Engagement in K-12 AI Education with a Card Game Construction Toolkit
abstract
With the growing prevalence of AI, the need for K-12 AI education is becoming more crucial, which is prompting active research in developing engaging AI learning activities. In this paper, we present our work on a game construction toolkit for middle school students and educators that enables them to tailor an AI-focused unplugged card game activity. In our prior work, we designed, developed, and piloted an unplugged card game activity where players predict the identity of a person based on hand-drawn features extracted from a set of facial cards. The activity aims to teach AI concepts aligned with one of the big ideas in AI utilizing techniques from facial recognition. During our pilot testing of the activity, we discovered that creating face cards that capture students' interest is a crucial factor in promoting student engagement. As a result, we designed a card game construction toolkit that allows students and educators to craft their own face card decks using photos that are personally interesting to them, looking to foster engagement and improve replayability of the activity. The toolkit's design is focused on ensuring easy accessibility and features a simple web-based interface that allows users to download and print their customized cards. We expect this toolkit will enhance the usability and educational effectiveness of our unplugged K-12 AI education activity.
Hansol Lim, Wookhee Min, Jessica Vandenberg, Veronica Cateté, Judith Uchidiuno, Bradford W. Mott
SIGCSE (2)2
2023 Enhancing Stealth Assessment in Collaborative Game-Based Learning with Multi-task Learning
Anisha Gupta, Dan Carpenter, Wookhee Min, Bradford W. Mott, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
AIED3
2023 Robust Team Communication Analytics with Transformer-Based Dialogue Modeling
Jay Pande, Wookhee Min, Randall Spain, Jason D. Saville, James C. Lester
AIED2
2023 Leveraging Game Design Activities for Middle Grades AI Education in Rural Communities
abstract
The ever pervasive nature of artificial intelligence (AI) in our world necessitates a focus on fostering an AI literate society. Young children, those aged 11 to 14, are at a critical point in developing their dispositions toward and perceptions of science, technology, engineering, and mathematics (STEM), which influences their future education and career interests. Youth in rural areas are in particular need of access to AI learning opportunities to prepare them for the future workforce; digital games may be one way to attract young, rural students to STEM education and careers. In this paper, we explore how to introduce rural middle grades students to foundational AI concepts through digital game design activities. To inform our efforts and to establish an understanding of what these student populations as well as their teachers know about AI and games, we conducted a set of interviews and focus groups. In brief, students’ awareness and understanding of AI varied significantly, whereas teachers had limited knowledge of AI. Moreover, students shared great interest in playing and designing games. In support of our findings, we are developing a set of game design activities around five core AI concepts and ensuring the activities are of interest to our rural students.
Jessica Vandenberg, Wookhee Min, Veronica Cateté, Danielle Boulden, Bradford W. Mott
FDG2
2023 Toward AI-infused Game Design Activities for Rural Middle Grades Students
abstract
The ubiquity of artificial intelligence (AI) in everyday life suggests the need to ensure young students know about AI, its uses and limitations, and its benefits and risks, while enabling them to develop expertise in using AI-driven technologies. To support rural middle grades students and educators in learning and teaching AI concepts, we are designing AI-focused learning activities centered around the creation of digital gameplay experiences. To inform our designs, we conducted educator interviews and student focus groups to gain insights into their understanding of AI, their computer science background, and their knowledge and interest in gaming. Building on findings from these interviews and focus groups, we have designed a set of hands-on activities to elicit deeper feedback from students and educators on their preferences, points of confusion, and interests. In this work, we present our initial AI-infused game design activities.
Jessica Vandenberg, Wookhee Min, Anisha Gupta, Veronica Cateté, Danielle Boulden, Bradford W. Mott
ITiCSE (2)2
2023 Multimodal Predictive Student Modeling with Multi-Task Transfer Learning
abstract
Game-based learning environments have the distinctive capacity to promote learning experiences that are both engaging and effective. Recent advances in sensor-based technologies (e.g., facial expression analysis and eye gaze tracking) and natural language processing have introduced the opportunity to leverage multimodal data streams for learning analytics. Learning analytics and student modeling informed by multimodal data captured during students’ interactions with game-based learning environments hold significant promise for designing effective learning environments that detect unproductive student behaviors and provide adaptive support for students during learning. Learning analytics frameworks that can accurately predict student learning outcomes early in students’ interactions hold considerable promise for enabling environments to dynamically adapt to individual student needs. In this paper, we investigate a multimodal, multi-task predictive student modeling framework for game-based learning environments. The framework is evaluated on two datasets of game-based learning interactions from two student populations (n=61 and n=118) who interacted with two versions of a game-based learning environment for microbiology education. The framework leverages available multimodal data channels from the datasets to simultaneously predict student post-test performance and interest. In addition to inducing models for each dataset individually, this work investigates the ability to use information learned from one source dataset to improve models based on another target dataset (i.e., transfer learning using pre-trained models). Results from a series of ablation experiments indicate the differences in predictive capacity among a combination of modalities including gameplay, eye gaze, facial expressions, and reflection text for predicting the two target variables. In addition, multi-task models were able to improve predictive performance compared to single-task baselines for one target variable, but not both. Lastly, transfer learning showed promise in improving predictive capacity in both datasets.
Andrew Emerson, Wookhee Min, Jonathan P. Rowe, Roger Azevedo, James C. Lester
LAK2
2023 Promoting AI Education for Rural Middle Grades Students with Digital Game Design
abstract
The demand is growing for a populace that is literate in Artificial Intelligence (AI); such literacy centers on enabling individuals to evaluate, collaborate with, and effectively use AI. Because the middle school years are a critical time for developing youths' perceptions and dispositions toward STEM, creating engaging AI learning experiences for middle grades students (ages 11 to 14) is paramount. The need for providing enhanced access to AI learning opportunities is especially pronounced in rural areas, which are typically underserved and underresourced. Inspired by prior research that game design holds significant potential for cultivating student interest and knowledge in computer science, we are designing, developing, and iteratively refining an AI-centered development environment that infuses AI learning into game design activities. In this work, we review design principles for game design interventions focused on middle grades computer science education and explore how to introduce AI learning experiences into interactive game design activities. We also discuss results from our initial co-design sessions with middle grades students and teachers in rural communities.
Jessica Vandenberg, Wookhee Min, Veronica Cateté, Danielle Boulden, Bradford W. Mott
SIGCSE (2)2
2022 Enhancing Stealth Assessment in Game-Based Learning Environments with Generative Zero-Shot Learning
Nathan L. Henderson, Halim Acosta, Wookhee Min, Bradford W. Mott, Trudi Lord, Frieda Reichsman, Chad Dorsey, Eric N. Wiebe, James C. Lester
EDM3
2022 Disruptive Talk Detection in Multi-Party Dialogue within Collaborative Learning Environments with a Regularized User-Aware Network
abstract
Kyungjin Park, Hyunwoo Sohn, Wookhee Min, Bradford Mott, Krista Glazewski, Cindy E. Hmelo-Silver, James Lester. Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue. 2022.
Kyungjin Park, Hyunwoo Sohn, Wookhee Min, Bradford W. Mott, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
SIGDIAL3
2021 Enhancing Multimodal Affect Recognition with Multi-Task Affective Dynamics Modeling
abstract
Accurately recognizing students’ affective states is critical for enabling adaptive learning environments to promote engagement and enhance learning outcomes. Multimodal approaches to student affect recognition capture multi-dimensional patterns of student behavior through the use of multiple data channels. An important factor in multimodal affect recognition is the context in which affect is experienced and exhibited. In this paper, we present a multimodal, multitask affect recognition framework that predicts students’ future affective states as auxiliary training tasks and uses prior affective states as input features to capture bi-directional affective dynamics and enhance the training of affect recognition models. Additionally, we investigate cross-stitch networks to maintain parameterized separation between shared and task-specific representations and task-specific uncertainty-weighted loss functions for contextual modeling of student affective states. We evaluate our approach using interaction and posture data captured from students engaged with a game-based learning environment for emergency medical training. Results indicate that the affective dynamics-based approach yields significant improvements in multimodal affect recognition across four different affective states.
Nathan L. Henderson, Wookhee Min, Jonathan P. Rowe, James C. Lester
ACII2
2021 Multimodal Trajectory Analysis of Visitor Engagement with Interactive Science Museum Exhibits
Andrew Emerson, Nathan L. Henderson, Wookhee Min, Jonathan P. Rowe, James Minogue, James C. Lester
AIED (2)3
2021 Multidimensional Team Communication Modeling for Adaptive Team Training: A Hybrid Deep Learning and Graphical Modeling Framework
Wookhee Min, Randall Spain, Jason D. Saville, Bradford W. Mott, Keith W. Brawner, Joan Hall Johnston, James C. Lester
AIED (1)1
2021 Early Prediction of Museum Visitor Engagement with Multimodal Adversarial Domain Adaptation
Nathan L. Henderson, Wookhee Min, Andrew Emerson, Jonathan P. Rowe, Seung Y. Lee, James Minogue, James C. Lester
EDM2
2021 What's Fair is Fair: Detecting and Mitigating Encoded Bias in Multimodal Models of Museum Visitor Attention
abstract
Recent years have seen growing interest in modeling visitor engagement in museums with multimodal learning analytics. In parallel, there has also been growing concern about issues of fairness and encoded bias in machine learning models. In this paper, we investigate bias detection and mitigation techniques to address issues of algorithmic fairness in multimodal models of museum visitor visual attention. We employ slicing analysis using the Absolute Between-ROC Area (ABROCA) statistic to detect encoded bias present in multimodal models of visitor visual attention trained with facial expression and posture data from visitor interactions with a game-based museum exhibit about environmental sustainability. We investigate instances of gender bias that arise between different combinations of modalities across several machine learning techniques. We also measure the effectiveness of two different debiasing strategies—learned fair representations and reweighing—when applied to the trained multimodal visitor attention models. Results indicate that patterns of bias can arise across different modality combinations for the different visitor visual attention models, and there is often an inherent tradeoff between predictive accuracy and ABROCA. Analyses suggest that debiasing strategies tend to be more effective on multimodal models of visitor visual attention than their unimodal counterparts
Halim Acosta, Nathan L. Henderson, Jonathan P. Rowe, Wookhee Min, James Minogue, James C. Lester
ICMI4
2021 Detecting Disruptive Talk in Student Chat-Based Discussion within Collaborative Game-Based Learning Environments
abstract
Collaborative game-based learning environments offer significant promise for creating engaging group learning experiences. Online chat plays a pivotal role in these environments by providing students with a means to freely communicate during problem solving. These chat-based discussions and negotiations support the coordination of students’ in-game learning activities. However, this freedom of expression comes with the possibility that some students might engage in undesirable communicative behavior. A key challenge posed by collaborative game-based learning environments is how to reliably detect disruptive talk that purposefully disrupt team dynamics and problem-solving interactions. Detecting disruptive talk during collaborative game-based learning is particularly important because if it is allowed to persist, it can generate frustration and significantly impede the learning process for students. This paper analyzes disruptive talk in a collaborative game-based learning environment for middle school science education to investigate how such behaviors influence students’ learning outcomes and varies across gender and students’ prior knowledge. We present a disruptive talk detection framework that automatically detects disruptive talk in chat-based group conversations. We further investigate both classic machine learning and deep learning models for the framework utilizing a range of dialogue representations as well as supplementary information such as student gender. Findings show that long short-term memory network (LSTM)-based disruptive talk detection models outperform competitive baseline models, indicating that the LSTM-based disruptive talk detection framework offers significant potential for supporting effective collaborative game-based learning through the identification of disruptive talk.
Kyungjin Park, Hyunwoo Sohn, Bradford W. Mott, Wookhee Min, Asmalina Saleh, Krista D. Glazewski, Cindy E. Hmelo-Silver, James C. Lester
LAK4
2020 Investigating Visitor Engagement in Interactive Science Museum Exhibits with Multimodal Bayesian Hierarchical Models
Andrew Emerson, Nathan L. Henderson, Jonathan P. Rowe, Wookhee Min, Seung Y. Lee, James Minogue, James C. Lester
AIED (1)4
2020 Generating Game Levels to Develop Computer Science Competencies in Game-Based Learning Environments
Kyungjin Park, Bradford W. Mott, Wookhee Min, Eric N. Wiebe, Kristy Elizabeth Boyer, James C. Lester
AIED (2)3
2020 Promoting Computer Science Learning with Block-Based Programming and Narrative-Centered Gameplay
abstract
Recent years have seen increasing awareness of the need for all students in primary and secondary education to learn computer science (CS) concepts and skills. Educational games hold significant potential to serve as a platform for CS education because they integrate engaging problem solving with effective pedagogical strategies. This potential is especially high for narrative-centered educational games that embed learning activities within rich interactive stories. In this paper, we present an educational game featuring block-based programming challenges contextualized within an engaging narrative, designed to promote CS learning for middle school students (ages 11 to 13). In the game, students undertake problem-solving challenges that are aligned with the K-12 Computer Science Framework. Results from a classroom implementation of the game with middle grade students suggest that their perceived game control ratings are positively correlated with their progress in the game, which suggests the need for adaptively supporting students' game-based learning activities. Building on these findings, we discuss design implications for creating student-adaptive CS learning experiences in educational games that incorporate block-based programming enriched narrative-centered gameplay.
Wookhee Min, Bradford W. Mott, Kyungjin Park, Sandra Taylor, Bita Akram, Eric N. Wiebe, Kristy Elizabeth Boyer, James C. Lester
CoG1
2020 Automated Assessment of Computer Science Competencies from Student Programs with Gaussian Process Regression
Bita Akram, Hamoon Azizsoltani, Wookhee Min, Eric N. Wiebe, Bradford W. Mott, Anam Navied, Kristy Elizabeth Boyer, James C. Lester
EDM3
2020 Enhancing Student Competency Models for Game-Based Learning with a Hybrid Stealth Assessment Framework
Nathan L. Henderson, Vikram Kumara, Wookhee Min, Bradford W. Mott, Danielle Boulden, Trudi Lord, Frieda Reichsman, Chad Dorsey, Eric N. Wiebe, James C. Lester
EDM3
2020 Early Prediction of Visitor Engagement in Science Museums with Multimodal Learning Analytics
abstract
Modeling visitor engagement is a key challenge in informal learning environments, such as museums and science centers. Devising predictive models of visitor engagement that accurately forecast salient features of visitor behavior, such as dwell time, holds significant potential for enabling adaptive learning environments and visitor analytics for museums and science centers. In this paper, we introduce a multimodal early prediction approach to modeling visitor engagement with interactive science museum exhibits. We utilize multimodal sensor data including eye gaze, facial expression, posture, and interaction log data captured during visitor interactions with an interactive museum exhibit for environmental science education, to induce predictive models of visitor dwell time. We investigate machine learning techniques (random forest, support vector machine, Lasso regression, gradient boosting trees, and multi-layer perceptron) to induce multimodal predictive models of visitor engagement with data from 85 museum visitors. Results from a series of ablation experiments suggest that incorporating additional modalities into predictive models of visitor engagement improves model accuracy. In addition, the models show improved predictive performance over time, demonstrating that increasingly accurate predictions of visitor dwell time can be achieved as more evidence becomes available from visitor interactions with interactive science museum exhibits. These findings highlight the efficacy of multimodal data for modeling museum exhibit visitor engagement.
Andrew Emerson, Nathan L. Henderson, Jonathan P. Rowe, Wookhee Min, Seung Y. Lee, James Minogue, James C. Lester
ICMI4
2020 Enhancing Affect Detection in Game-Based Learning Environments with Multimodal Conditional Generative Modeling
abstract
Accurately detecting and responding to student affect is a critical capability for adaptive learning environments. Recent years have seen growing interest in modeling student affect with multimodal sensor data. A key challenge in multimodal affect detection is dealing with data loss due to noisy, missing, or invalid multimodal features. Because multimodal affect detection often requires large quantities of data, data loss can have a strong, adverse impact on affect detector performance. To address this issue, we present a multimodal data imputation framework that utilizes conditional generative models to automatically impute posture and interaction log data from student interactions with a game-based learning environment for emergency medical training. We investigate two generative models, a Conditional Generative Adversarial Network (C-GAN) and a Conditional Variational Autoencoder (C-VAE), that are trained using a modality that has undergone varying levels of artificial data masking. The generative models are conditioned on the corresponding intact modality, enabling the data imputation process to capture the interaction between the concurrent modalities. We examine the effectiveness of the conditional generative models on imputation accuracy and its impact on the performance of affect detection. Each imputation model is evaluated using varying amounts of artificial data masking to determine how the data missingness impacts the performance of each imputation method. Results based on the modalities captured from students? interactions with the game-based learning environment indicate that deep conditional generative models within a multimodal data imputation framework yield significant benefits compared to baseline imputation techniques in terms of both imputation accuracy and affective detector performance.
Nathan L. Henderson, Wookhee Min, Jonathan P. Rowe, James C. Lester
ICMI2
2020 A Conceptual Assessment Framework for K-12 Computer Science Rubric Design
abstract
The lack of effective guidelines for assessing students' computer science (CS) competencies is creating significant demand by K-12 teachers for CS assessments to evaluate students' learning. We propose a conceptual assessment framework that guides teachers through designing appropriate assessments for computer science (CS) activities in their classrooms. The framework addresses the critical problem of incorporating CS into K-12 curricula without corresponding assessments. We illustrate its use with the design of a rubric for a bubble sort algorithm situated in a game-based learning environment for middle-grade students. We also apply a preliminary and a revised version of this assessment on two datasets collected from students' interactions with the learning environment. We found consistency among results identified through applying the preliminary and the revised rubric. The results reveal distinctive patterns in students' approaches to CS problem solving and coherency with respect to different aspects of the rubric.*
Bita Akram, Wookhee Min, Eric N. Wiebe, Anam Navied, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
SIGCSE2
2019 Predicting Dialogue Breakdown in Conversational Pedagogical Agents with Multimodal LSTMs
Wookhee Min, Kyungjin Park, Joseph B. Wiggins, Bradford W. Mott, Eric N. Wiebe, Kristy Elizabeth Boyer, James C. Lester
AIED (2)1
2019 Take the Initiative: Mixed Initiative Dialogue Policies for Pedagogical Agents in Game-Based Learning Environments
Joseph B. Wiggins, Mayank Kulkarni, Wookhee Min, Kristy Elizabeth Boyer, Bradford W. Mott, Eric N. Wiebe, James C. Lester
AIED (2)3
2019 Generating Educational Game Levels with Multistep Deep Convolutional Generative Adversarial Networks
abstract
Educational games offer significant potential for supporting personalized learning in engaging virtual worlds. However, many educational games do not provide adaptive gameplay to meet the needs of individual students. To address this issue, educational games should include game levels that can self-adjust to the specific needs of individual students. However, creating a large number of adaptable game levels requires considerable effort by game developers. A promising solution to this problem is to leverage procedural content generation to automatically generate levels for educational games that incorporate the desired learning objectives. In this paper, we propose a multistep deep convolutional generative adversarial network for generating new levels within a game for middle school computer science education. The model operates in two phases: (1) train a generator with a small set of human-authored example levels and generate a much larger set of synthetic levels to augment the training data for a second generator, and (2) train a second generator using the augmented training data and use it to generate novel educational game levels with enhanced solvability. We evaluate the performance of the model by comparing the novelty and solvability of generated levels between the two generators. Results suggest that the proposed multistep model significantly enhances the solvability of the generated levels with only minor degradation in the novelty of the generated levels.
Kyungjin Park, Bradford W. Mott, Wookhee Min, Kristy Elizabeth Boyer, Eric N. Wiebe, James C. Lester
CoG3
2019 Predicting Early and Often: Predictive Student Modeling for Block-Based Programming Environments
Andrew Emerson, Andy Smith, Cody Smith, Fernando J. Rodríguez, Wookhee Min, Eric N. Wiebe, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
EDM5
2019 Assessing Middle School Students' Computational Thinking Through Programming Trajectory Analysis
abstract
With national K-12 education initiatives such as "CSForAll," block-based programming environments have emerged as widely used tools for teaching novice programming. A key challenge presented by block-based programming environments is assessing students' computational thinking (CT) and programming competencies. Developing assessment methods that can evaluate students' use of CT practices such as testing and refining, and developing and using appropriate algorithms, can help teachers evaluate students learning and provide appropriate scaffolding. In this work, we utilize an evidence-centered assessment design approach to devise a three-dimensional assessment to evaluate students' CT competencies based on evidence extracted from their programming trajectories in a block-based programming environment. In this assessment, the first dimension assesses students' knowledge of essential CT concepts, the second dimension assesses students' dynamic testing and refining strategies, and the third dimension assesses their overall problem-solving efficiency. We apply the assessment framework to data collected from students' interactions with a game-based learning environment designed to develop middle-grade students' CT competencies and programming skills. The results demonstrate that students' knowledge of basic CT constructs, such as appropriate use and combination of control structures, serves as the foundation for designing and implementing effective algorithms. Further, we assessed students testing and refining strategies over the three dimensions of novelty, positivity, and scale. The results demonstrate that students with higher algorithmic capabilities tend to make more novel, positive, and small-scale changes. The results reveal distinctive patterns in students' approaches to computational thinking problem solving and make a step toward identifying and assessing productive computational thinking practices.
Bita Akram, Wookhee Min, Eric N. Wiebe, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
SIGCSE2
2018 Improving Stealth Assessment in Game-based Learning with LSTM-based Analytics
Bita Akram, Wookhee Min, Eric N. Wiebe, Bradford W. Mott, Kristy Elizabeth Boyer, James C. Lester
EDM2
2018 High-Fidelity Simulated Players for Interactive Narrative Planning
abstract
Interactive narrative planning offers significant potential for creating adaptive gameplay experiences. While data-driven techniques have been devised that utilize player interaction data to induce policies for interactive narrative planners, they require enormously large gameplay datasets. A promising approach to addressing this challenge is creating simulated players whose behaviors closely approximate those of human players. In this paper, we propose a novel approach to generating high-fidelity simulated players based on deep recurrent highway networks and deep convolutional networks. Empirical results demonstrate that the proposed models significantly outperform the prior state-of-the-art in generating high-fidelity simulated player models that accurately imitate human players’ narrative interactions. Using the high-fidelity simulated player models, we show the advantage of more exploratory reinforcement learning methods for deriving generalizable narrative adaptation policies.
Jonathan P. Rowe, Wookhee Min, Bradford W. Mott, James C. Lester
IJCAI3
2017 Inducing Stealth Assessors from Game Interaction Data
Wookhee Min, Megan Hardy Frankosky, Bradford W. Mott, Eric N. Wiebe, Kristy Elizabeth Boyer, James C. Lester
AIED1
2017 "Thanks Alisha, Keep in Touch": Gender Effects and Engagement with Virtual Learning Companions
Lydia Pezzullo, Joseph B. Wiggins, Megan Hardy Frankosky, Wookhee Min, Kristy Elizabeth Boyer, Bradford W. Mott, Eric N. Wiebe, James C. Lester
AIED4
2017 Interactive Narrative Personalization with Deep Reinforcement Learning
abstract
Data-driven techniques for interactive narrative generation are the subject of growing interest. Reinforcement learning (RL) offers significant potential for devising data-driven interactive narrative generators that tailor players’ story experiences by inducing policies from player interaction logs. A key open question in RL-based interactive narrative generation is how to model complex player interaction patterns to learn effective policies. In this paper we present a deep RL-based interactive narrative generation framework that leverages synthetic data produced by a bipartite simulated player model. Specifically, the framework involves training a set of Q-networks to control adaptable narrative event sequences with long short-term memory network-based simulated players. We investigate the deep RL framework’s performance with an educational interactive narrative, Crystal Island. Results suggest that the deep RL-based narrative generation framework yields effective personalized interactive narratives.
Jonathan P. Rowe, Wookhee Min, Bradford W. Mott, James C. Lester
IJCAI3
2016 Predicting Dialogue Acts for Intelligent Virtual Agents with Multimodal Student Interaction Data
Wookhee Min, Joseph B. Wiggins, Lydia Pezzullo, Alexandria K. Vail, Kristy Elizabeth Boyer, Bradford W. Mott, Megan Hardy Frankosky, Eric N. Wiebe, James C. Lester
EDM1
2016 Player Goal Recognition in Open-World Digital Games with Long Short-Term Memory Networks
Wookhee Min, Bradford W. Mott, Jonathan P. Rowe, Barry Liu, James C. Lester
IJCAI1
2016 Integrating Real-Time Drawing and Writing Diagnostic Models: An Evidence-Centered Design Framework for Multimodal Science Assessment
Andy Smith, Osman Aksit, Wookhee Min, Eric N. Wiebe, Bradford W. Mott, James C. Lester
ITS3
2015 DeepStealth: Leveraging Deep Learning Models for Stealth Assessment in Game-Based Learning Environments
Wookhee Min, Megan Hardy Frankosky, Bradford W. Mott, Jonathan P. Rowe, Eric N. Wiebe, Kristy Elizabeth Boyer, James C. Lester
AIED1
2015 ENGAGE: A Game-based Learning Environment for Middle School Computational Thinking
abstract
We present ENGAGE, a game-based learning environment for teaching computational thinking to middle school students. This project has dual aims: introducing computational thinking practices to students at a young age, and improving computational thinking attitudes among underrepresented students. In pursuit of these two goals, the ENGAGE team has mapped the learning objectives of the AP CS Principles course to the middle school level, and then built an immersive game experience upon that foundation. Students choose computer scientist avatars to represent themselves, and then play in pairs as they investigate a data-related mystery in an underwater research station, solving computational thinking challenges along the way. ENGAGE is currently being implemented as part of a quarterly elective in four middle schools in North Carolina. During the elective, students spend a total of ten classroom sessions playing the game, supplemented by "unplugged" activities that reinforce concepts learned in the game environment. We plan to expand to more middle schools in the 2015-2016 school year. In this demo, members of the SIGCSE community will be able to experience the ENGAGE game for themselves and learn more about its development and future directions. We will also discuss our success in recruiting and teaching the ENGAGE curriculum to middle school teachers who had no prior computer science experience, and the success of those middle school teachers in implementing ENGAGE within their classrooms.
Kristy Elizabeth Boyer, Philip Sheridan Buffum, Kirby Culbertson, Megan Hardy Frankosky, James C. Lester, Allison G. Martínez-Arocho, Wookhee Min, Bradford W. Mott, Fernando J. Rodríguez, Eric N. Wiebe
SIGCSE7
2015 Diagrammatic Student Models: Modeling Student Drawing Performance with Deep Learning
Andy Smith, Wookhee Min, Bradford W. Mott, James C. Lester
UMAP2
2014 FLARE: An open source toolkit for creating expressive user interfaces for serious games
Bradford W. Mott, Jonathan P. Rowe, Wookhee Min, Robert G. Taylor, James C. Lester
FDG3
2014 Leveraging Semi-Supervised Learning to Predict Student Problem-Solving Performance in Narrative-Centered Learning Environments
Wookhee Min, Bradford W. Mott, Jonathan P. Rowe, James C. Lester
Intelligent Tutoring Systems1
2013 Personalizing Embedded Assessment Sequences in Narrative-Centered Learning Environments: A Collaborative Filtering Approach
Wookhee Min, Jonathan P. Rowe, Bradford W. Mott, James C. Lester
AIED1
2008 PRISM: A Framework for Authoring Interactive Narratives
Yun-Gyung Cheong, Yeo-Jin Kim, Wookhee Min, Eok-Soo Shim
ICIDS3