For more than a decade, a handful of theoretical models have shaped a substantial amount of the research related to students’ emotional experiences during learning. This research has been productive, but articulating the underlying implicit assumptions in existing theories and their implications in our empirical interpretations can help to better investigate the reciprocal relationships between learning and emotion, and subsequently, to develop better interventions. This paper expands upon the existing theoretical frameworks, increasing the types of questions we ask about affect dynamics. We do so within the context of Crystal Island, a virtual world that allows middle school students to investigate microbiology questions. Specifically, we use this data to examine and revise the assumptions that are implicit in these models and the methods we use to investigate them.

Student affect is widely recognized as a major influence on learning gains and engagement, which has led to the development of many automated affect detectors. However, in order to respond effectively to student affect, we must know how students interpret it. This study proposes a novel automated detector that models when students attribute their epistemic emotion to task difficulty. The goal is to use detectors like this one to better understand how to respond to students' affective states (in this case, boredom, confusion, frustration and nervousness). We then discuss the implications of this novel detector for real-time support in game-based learning environments.

Pedagogical planners can provide adaptive support to students in narrative-centered learning environments by dynamically scaffolding student learning and tailoring problem scenarios. Reinforcement learning (RL) is frequently used for pedagogical planning in narrative-centered learning environments. However, RL-based pedagogical planning raises significant challenges due to the scarcity of data for training RL policies. Most prior work has relied on limited-size datasets and offline RL techniques for policy learning. Unfortunately, offline RL techniques do not support on-demand exploration and evaluation, which can adversely impact the quality of induced policies. To address the limitation of data scarcity and offline RL, we propose INSIGHT, an online RL framework for training data-driven pedagogical policies that optimize student learning in narrative-centered learning environments. The INSIGHT framework consists of three components: a narrative-centered learning environment simulator, a simulated student agent, and an RL-based pedagogical planner agent, which uses a reward metric that is associated with effective student learning processes. The framework enables the generation of synthetic data for on-demand exploration and evaluation of RL-based pedagogical planning. We have implemented INSIGHT with OpenAI Gym for a narrative-centered learning environment testbed with rule-based simulated student agents and a deep Q-learning-based pedagogical planner. Our results show that online deep RL algorithms can induce near-optimal pedagogical policies in the INSIGHT framework, while offline deep RL algorithms only find suboptimal policies even with large amounts of data.

In the evolving field of mixed-initiative game design, where procedural content generation plays a pivotal role, establishing a comprehensive approach that empowers non-technical designers to actively shape content generation is essential. Recent developments in large language models significantly alter the landscape of automated text-based content generation. These models offer a significant advantage in mixed-initiative procedural level generation by providing designers with intuitive, natural language interfaces. The framework presented in this paper interprets natural language inputs, detailing level design constraints and optimization goals, to aid in the cooperative development of game levels for a strategy game aimed at environmental sustainability education. It enables designers to articulate their vision concerning the problem domain, goal metrics, and desired difficulty level through a textual description. By utilizing large language models, the framework extracts semantic constraints and optimization objectives, which are then used to generate candidate game levels. The efficacy of these levels is assessed by game-playing agents trained through advanced deep reinforcement learning methods, ensuring alignment with the designer's original specifications. We further evaluate our framework with both experts and non-experts in designing levels for our strategy game. Their detailed responses confirm that our framework effectively translates natural language descriptions into playable game levels, accurately capturing the designers' intended objectives.

As the role of procedural content generation in mixed-initiative game design continues to grow, it is crucial to develop an end-to-end approach that enables non-technical designers to artfully guide content generation. Recent advances in large language models, such as GPT-4, are rapidly transforming the landscape of automated generation of text-based content. Large language models have significant potential for mixed-initiative procedural level generation by providing natural language interfaces for designers. This paper presents an end-to-end procedural level generation framework that interprets natural language descriptions of level design constraints and optimization objectives to facilitate the collaborative creation of game levels for a strategy game focused on environmental sustainability education. The framework enables designers to specify a problem domain, goal metrics, and target difficulty via natural language description. It then employs large language models for the semantic extraction of constraints and optimization targets to drive the generation of candidate levels. Generated game levels are evaluated via game-playing agents trained with deep reinforcement learning techniques to ensure the game levels meet the level designer’s specifications. Manual evaluation by authors shows that the proposed framework can effectively transform designers’ natural language descriptions into fully playable game levels that reflect their intended design objectives.

Game-based learning environments have the distinctive capacity to promote learning experiences that are both engaging and effective. Recent advances in sensor-based technologies (e.g., facial expression analysis and eye gaze tracking) and natural language processing have introduced the opportunity to leverage multimodal data streams for learning analytics. Learning analytics and student modeling informed by multimodal data captured during students’ interactions with game-based learning environments hold significant promise for designing effective learning environments that detect unproductive student behaviors and provide adaptive support for students during learning. Learning analytics frameworks that can accurately predict student learning outcomes early in students’ interactions hold considerable promise for enabling environments to dynamically adapt to individual student needs. In this paper, we investigate a multimodal, multi-task predictive student modeling framework for game-based learning environments. The framework is evaluated on two datasets of game-based learning interactions from two student populations (n=61 and n=118) who interacted with two versions of a game-based learning environment for microbiology education. The framework leverages available multimodal data channels from the datasets to simultaneously predict student post-test performance and interest. In addition to inducing models for each dataset individually, this work investigates the ability to use information learned from one source dataset to improve models based on another target dataset (i.e., transfer learning using pre-trained models). Results from a series of ablation experiments indicate the differences in predictive capacity among a combination of modalities including gameplay, eye gaze, facial expressions, and reflection text for predicting the two target variables. In addition, multi-task models were able to improve predictive performance compared to single-task baselines for one target variable, but not both. Lastly, transfer learning showed promise in improving predictive capacity in both datasets.

Artificial intelligence has emerged as a technology that is profoundly reshaping society and enabling rapid improvements in science, engineering, and mathematics, as well as information technology itself. This has generated increased demand for fostering an AI-literate populace as well as a growing recognition of the importance of promoting K-12 students’ awareness and interest in AI. Although efforts are be-ginning to incorporate AI learning within K-12 education, there is little research exploring how to introduce students to AI and how to support teachers to integrate AI learning experiences in their classrooms. This is especially true at the elementary school level. A particularly promising approach for providing effective and engaging AI learning experiences for elementary students is game-based learning. In this paper, we explore how to introduce AI-infused collaborative inquiry learning into upper elementary school (student ages 8 to 11) using game-based learning. To ground the work in the realities of elementary school classrooms, we present insights from interviews with elementary school teachers to under-stand how best to support them in integrating AI into their classrooms. We then present the design of PrimaryAI, a game-based learning environment that supports rich problem-based learning activities within upper elementary classrooms centered on AI applied toward solving life-science problems. Finally, we discuss some of the challenges we face in bringing AI-infused collaborative inquiry learning to upper elementary students.

Accurately recognizing students’ affective states is critical for enabling adaptive learning environments to promote engagement and enhance learning outcomes. Multimodal approaches to student affect recognition capture multi-dimensional patterns of student behavior through the use of multiple data channels. An important factor in multimodal affect recognition is the context in which affect is experienced and exhibited. In this paper, we present a multimodal, multitask affect recognition framework that predicts students’ future affective states as auxiliary training tasks and uses prior affective states as input features to capture bi-directional affective dynamics and enhance the training of affect recognition models. Additionally, we investigate cross-stitch networks to maintain parameterized separation between shared and task-specific representations and task-specific uncertainty-weighted loss functions for contextual modeling of student affective states. We evaluate our approach using interaction and posture data captured from students engaged with a game-based learning environment for emergency medical training. Results indicate that the affective dynamics-based approach yields significant improvements in multimodal affect recognition across four different affective states.

Recent years have seen growing interest in modeling visitor engagement in museums with multimodal learning analytics. In parallel, there has also been growing concern about issues of fairness and encoded bias in machine learning models. In this paper, we investigate bias detection and mitigation techniques to address issues of algorithmic fairness in multimodal models of museum visitor visual attention. We employ slicing analysis using the Absolute Between-ROC Area (ABROCA) statistic to detect encoded bias present in multimodal models of visitor visual attention trained with facial expression and posture data from visitor interactions with a game-based museum exhibit about environmental sustainability. We investigate instances of gender bias that arise between different combinations of modalities across several machine learning techniques. We also measure the effectiveness of two different debiasing strategies—learned fair representations and reweighing—when applied to the trained multimodal visitor attention models. Results indicate that patterns of bias can arise across different modality combinations for the different visitor visual attention models, and there is often an inherent tradeoff between predictive accuracy and ABROCA. Analyses suggest that debiasing strategies tend to be more effective on multimodal models of visitor visual attention than their unimodal counterparts

Reflection plays a critical role in learning by encouraging students to contemplate their knowledge and previous learning experiences to inform their future actions and higher-order thinking, such as reasoning and problem solving. Reflection is particularly important in inquiry-driven learning scenarios where students have the freedom to set goals and regulate their own learning. However, despite the importance of reflection in learning, there are significant theoretical, methodological, and analytical challenges posed by measuring, modeling, and supporting reflection. This paper presents results from a classroom study to investigate middle-school students’ reflection during inquiry-driven learning with Crystal Island, a game-based learning environment for middle-school microbiology. To collect evidence of reflection during game-based learning, we used embedded reflection prompts to elicit written reflections during students’ interactions with Crystal Island. Results from analysis of data from 105 students highlight relationships between features of students’ reflections and learning outcomes related to both science content knowledge and problem solving. We consider implications for building adaptive support in game-based learning environments to foster deep reflection and enhance learning, and we identify key features in students’ problem-solving actions and reflections that are predictive of reflection depth. These findings present a foundation for providing adaptive support for reflection during game-based learning.

Modeling student knowledge is critical in adaptive learning environments. Predictive student modeling enables formative assessment of student knowledge and skills, and it drives personalized support to create learning experiences that are both effective and engaging. Traditional approaches to predictive student modeling utilize features extracted from students’ interaction trace data to predict student test performance, aggregating student test performance as a single output label. We reformulate predictive student modeling as a multi-task learning problem, modeling questions from student test data as distinct “tasks.” We demonstrate the effectiveness of this approach by utilizing student data from a series of laboratory-based and classroom-based studies conducted with a game-based learning environment for microbiology education, Crystal Island. Using sequential representations of student gameplay, results show that multi-task stacked LSTMs with residual connections significantly outperform baseline models that do not use the multi-task formulation. Additionally, the accuracy of predictive student models is improved as the number of tasks increases. These findings have significant implications for the design and development of predictive student models in adaptive learning environments.

Modeling visitor engagement is a key challenge in informal learning environments, such as museums and science centers. Devising predictive models of visitor engagement that accurately forecast salient features of visitor behavior, such as dwell time, holds significant potential for enabling adaptive learning environments and visitor analytics for museums and science centers. In this paper, we introduce a multimodal early prediction approach to modeling visitor engagement with interactive science museum exhibits. We utilize multimodal sensor data including eye gaze, facial expression, posture, and interaction log data captured during visitor interactions with an interactive museum exhibit for environmental science education, to induce predictive models of visitor dwell time. We investigate machine learning techniques (random forest, support vector machine, Lasso regression, gradient boosting trees, and multi-layer perceptron) to induce multimodal predictive models of visitor engagement with data from 85 museum visitors. Results from a series of ablation experiments suggest that incorporating additional modalities into predictive models of visitor engagement improves model accuracy. In addition, the models show improved predictive performance over time, demonstrating that increasingly accurate predictions of visitor dwell time can be achieved as more evidence becomes available from visitor interactions with interactive science museum exhibits. These findings highlight the efficacy of multimodal data for modeling museum exhibit visitor engagement.

Accurately detecting and responding to student affect is a critical capability for adaptive learning environments. Recent years have seen growing interest in modeling student affect with multimodal sensor data. A key challenge in multimodal affect detection is dealing with data loss due to noisy, missing, or invalid multimodal features. Because multimodal affect detection often requires large quantities of data, data loss can have a strong, adverse impact on affect detector performance. To address this issue, we present a multimodal data imputation framework that utilizes conditional generative models to automatically impute posture and interaction log data from student interactions with a game-based learning environment for emergency medical training. We investigate two generative models, a Conditional Generative Adversarial Network (C-GAN) and a Conditional Variational Autoencoder (C-VAE), that are trained using a modality that has undergone varying levels of artificial data masking. The generative models are conditioned on the corresponding intact modality, enabling the data imputation process to capture the interaction between the concurrent modalities. We examine the effectiveness of the conditional generative models on imputation accuracy and its impact on the performance of affect detection. Each imputation model is evaluated using varying amounts of artificial data masking to determine how the data missingness impacts the performance of each imputation method. Results based on the modalities captured from students? interactions with the game-based learning environment indicate that deep conditional generative models within a multimodal data imputation framework yield significant benefits compared to baseline imputation techniques in terms of both imputation accuracy and affective detector performance.

Utilizing sensors for affect detection in adaptive learning technologies has been the subject of growing interest in recent years. This extends to the collection of multiple concurrent sensor-based input channels to enable multimodal affective modeling. However, sensors pose significant challenges to affect detection, including sensor connectivity issues, background noise, inconsistent data logging, and loss of data due to hardware failure. In this paper, we introduce a framework for multimodal data imputation to improve automated detection of student affect in adaptive learning technologies. Through the use of an autoencoder neural network trained on Microsoft Kinect-based posture data and electrodermal activity data with synthetic noise injection, we approximate missing values within the original dataset while still preserving the inter-related context between features when reconstructing the dataset. The reconstructed dataset can be used in conjunction with multimodal data fusion techniques to further boost affect detector accuracy. Results indicate that this framework improves the effectiveness of multimodal affect detectors when compared to unimodal baseline models, as well as models using baseline data imputation techniques such as mean imputation. Further, it maintains cross-modality information that influences the multimodal affect detectors' performance, as the approach also outperforms previous work using the latent representation of the imputed dataset as training data instead of a complete reconstruction of the original dataset's dimensionality.

Interactive narrative planning offers significant potential for creating adaptive gameplay experiences. While data-driven techniques have been devised that utilize player interaction data to induce policies for interactive narrative planners, they require enormously large gameplay datasets. A promising approach to addressing this challenge is creating simulated players whose behaviors closely approximate those of human players. In this paper, we propose a novel approach to generating high-fidelity simulated players based on deep recurrent highway networks and deep convolutional networks. Empirical results demonstrate that the proposed models significantly outperform the prior state-of-the-art in generating high-fidelity simulated player models that accurately imitate human players’ narrative interactions. Using the high-fidelity simulated player models, we show the advantage of more exploratory reinforcement learning methods for deriving generalizable narrative adaptation policies.

Affective support can play a central role in adaptive learning environments. Although virtual human tutors hold significant promise for providing affective support, a key open question is how a tutor's facial expressions can influence learners' performance. In this paper, we report on a study to examine the influence of a human tutor agent's facial expressions on learners' performance and emotions during learning. Results from the study suggest that learners' performance is significantly better when a human tutor agent facially expresses emotions that are congruent with the content relevancy. Results also suggest that learners facially express significantly more confusion when the human tutor agent provides incongruent facial expressions. These results can inform the design of virtual humans as pedagogical agents can inform the design of virtual humans as pedagogical agents and designing intelligent learner-agent interactions.

Data-driven techniques for interactive narrative generation are the subject of growing interest. Reinforcement learning (RL) offers significant potential for devising data-driven interactive narrative generators that tailor players’ story experiences by inducing policies from player interaction logs. A key open question in RL-based interactive narrative generation is how to model complex player interaction patterns to learn effective policies. In this paper we present a deep RL-based interactive narrative generation framework that leverages synthetic data produced by a bipartite simulated player model. Specifically, the framework involves training a set of Q-networks to control adaptable narrative event sequences with long short-term memory network-based simulated players. We investigate the deep RL framework’s performance with an educational interactive narrative, Crystal Island. Results suggest that the deep RL-based narrative generation framework yields effective personalized interactive narratives.

Recent years have seen a growing recognition of the role that affect plays in learning. Because game-based learning environments elicit a wide range of student affective states, affect-enhanced student modeling for game-based learning holds considerable promise. This paper introduces an affect-enhanced student modeling framework that leverages facial expression tracking for game-based learning. The affect-enhanced student modeling framework was used to generate predictive models of student learning and student engagement for students who interacted with CRYSTAL ISLAND, a game-based learning environment for microbiology education. Findings from the study reveal that the affect-enhanced student models significantly outperform baseline predictive student models that utilize the same gameplay traces but do not use facial expression tracking. The study also found that models based on individual facial action coding units are more effective than composite emotion models. The findings suggest that introducing facial expression tracking can improve the accuracy of student models, both for predicting student learning gains and also for predicting student engagement.

Computational models of interactive narrative offer significant potential for creating educational game experiences that are procedurally tailored to individual players and support learning. A key challenge posed by interactive narrative is devising effective director agent models that dynamically sequence story events according to players' actions and needs. In this paper, we describe a supervised machine-learning framework to model director agent strategies in an educational interactive narrative Crystal Island. Findings from two studies with human participants are reported. The first study utilized a Wizard-of-Oz paradigm where human “wizards” directed participants through Crystal Island's mystery storyline by dynamically controlling narrative events in the game environment. Interaction logs yielded training data for machine learning the conditional probabilities of a dynamic Bayesian network (DBN) model of the human wizards' directorial actions. Results indicate that the DBN model achieved significantly higher precision and recall than naive Bayes and bigram model techniques. In the second study, the DBN director agent model was incorporated into the runtime version of Crystal Island, and its impact on students' narrative-centered learning experiences was investigated. Results indicate that machine-learning director agent strategies from human demonstrations yield models that positively shape players' narrative-centered learning and problem-solving experiences.

Goal recognition in digital games involves inferring players’ goals from observed sequences of low-level player actions. Goal recognition models support player-adaptive digital games, which dynamically augment game events in response to player choices for a range of applications, including entertainment, training, and education. However, digital games pose significant challenges for goal recognition, such as exploratory actions and ill-defined goals. This paper presents a goal recognition framework based on Markov logic networks (MLNs). The model’s parameters are directly learned from a corpus that was collected from player interactions with a non-linear educational game. An empirical evaluation demonstrates that the MLN goal recognition framework accurately predicts players’ goals in a game environment with exploratory actions and ill-defined goals.

Narrative-centered learning environments are an important class of educational games that situate learning within rich story contexts. The work presented in this paper investigates individual differences in gameplay and learning during student interactions with a narrative-centered learning environment, Crystal Island. Findings reveal striking differences between high- and low-achieving science students in problem-solving effectiveness, attention to particular gameplay elements, learning gains and engagement ratings. High-achieving science students tended to demonstrate greater problem-solving efficiency, reported higher levels of interest and presence in the narrative environment, and demonstrated an increased focus on information gathering and information organization gameplay activities. Lower-achieving microbiology students gravitated toward novel gameplay elements, such as conversations with non-player characters and the use of laboratory testing equipment. The findings have implications for the design of broadly effective gameplay activities for narrative-centered learning environments, as well as investigations of scaffolding techniques to promote effective problem solving, improved learning outcomes and sustained engagement for all students.

Recent years have seen increasing interest in narrative-centered learning environments. However, the same qualities that make them engaging can also introduce seductive details that invite off-task behavior. This paper examines off-task behavior in the CRYSTAL ISLAND narrative-centered learning environment. Results from an empirical study examining the relationships between student test performance, individual differences, and off-task behavior are presented. The study found negative correlations between off-task behavior and test performance, as well as significant gender effects on the total amount of off-task behavior. Initial conclusions from a path analysis conducted on students' action sequences are also presented.

Recent years have seen a growing interest in the role that narrative can play in learning. With the emergence of narrative-centered learning environments that engage students by drawing them into rich interactions with compelling characters, we have begun to see the significant potential offered by immersive story-based learning experiences. In this paper we describe two studies that investigate the impact of empathetic characters on student perceptions of presence. A study was initially conducted with middle school students, and was then replicated with high school students. The results indicate that, for both populations, employing empathetic characters in narrative-centered learning environments significantly increases student perceptions of presence. The studies also reveal that empathetic characters contribute to a heightened sense of student involvement and control in learning situations.