Learning analytics research increasingly studies classroom learning with AI-based systems through rich contextual data from outside these systems, especially student-teacher interactions. One key challenge in leveraging such data is generating meaningful insights into effective teacher practices. Quantitative ethnography bears the potential to close this gap by combining multimodal data streams into networks of co-occurring behavior that drive insight into favorable learning conditions. The present study uses transmodal ordered network analysis to understand effective teacher practices in relationship to traditional metrics of in-system learning in a mathematics classroom working with AI tutors. Incorporating teacher practices captured by position tracking and human observation codes into modeling significantly improved the inference of how efficiently students improved in the AI tutor beyond a model with tutor log data features only. Comparing teacher practices by student learning rates, we find that students with low learning rates exhibited more hint use after monitoring. However, after an extended visit, students with low learning rates showed learning behavior similar to their high learning rate peers, achieving repeated correct attempts in the tutor. Observation notes suggest conceptual and procedural support differences can help explain visit effectiveness. Taken together, offering early conceptual support to students with low learning rates could make classroom practice with AI tutors more effective. This study advances the scientific understanding of effective teacher practice in classrooms learning with AI tutors and methodologies to make such practices visible.

Affect dynamics, the study of how affect develops and manifests over time, has become a popular area of research in affective computing for learning. In this article, we first provide a detailed analysis of prior affect dynamics studies, elaborating both their findings and the contextual and methodological differences between these studies. We then address methodological concerns that have not been previously addressed in the literature, discussing how various edge cases should be treated. Next, we present mathematical evidence that several past studies applied the transition metric (L) incorrectly - leading to invalid conclusions of statistical significance - and provide a corrected method. Using this corrected analysis method, we reanalyze ten past affect datasets collected in diverse contexts and synthesize the results, determining that the findings do not match the most popular theoretical model of affect dynamics. Instead, our results highlight the need to focus on cultural factors in future affect dynamics research.

Several emotional theories that inform the design of Virtual Learning Environments (VLEs) categorize affect as either positive or negative. However, the relationship between affect and learning appears to be more complex than that. Despite several empirical investigations in the last fifteen years, including a few that have attempted to complexify the role of affect in students’ learning in VLE, there has not been an attempt to synthesize the evidence across them. To bridge this gap, we conducted a systematic review of empirical studies that examined the relationship between student outcomes and the affect that arises during their interaction with a VLE. Our synthesis of results across thirty-nine papers suggests that except engagement, all of the commonly studied affective states (confusion, frustration, and boredom) have mixed relationships with outcomes. We further explored the differences in student demographics and study context to explain the variation in the results. Some of our key findings include poorer learning outcomes arising for confusion in classrooms (versus lab studies), differences in brief versus prolonged confusion and resolved versus persistent confusion, more positive (versus null) results for engagement in learning games, and more significant results for rarer affective states like frustration with automated affect detectors (versus student self-reports). We conclude that more careful attention must be paid to contextual differences in affect's role in student learning. We discuss the implication of this review for VLE design and research.

Despite the abundance of data generated from students’ activities in virtual learning environments, the use of supervised machine learning in learning analytics is limited by the availability of labeled data, which can be difficult to collect for complex educational constructs. In a previous study, a subfield of machine learning called Active Learning (AL) was explored to improve the data labeling efficiency. AL trains a model and uses it, in parallel, to choose the next data sample to get labeled from a human expert. Due to the complexity of educational constructs and data, AL has suffered from the cold-start problem where the model does not have access to sufficient data yet to choose the best next sample to learn from. In this paper, we explore the use of past data to warm start the AL training process. We also critically examine the implications of differing contexts (urbanicity) in which the past data was collected. To this end, we use authentic affect labels collected through human observations in middle school mathematics classrooms to simulate the development of AL-based detectors of engaged concentration. We experiment with two AL methods (uncertainty sampling, L-MMSE) and random sampling for data selection. Our results suggest that using past data to warm start AL training could be effective for some methods based on the target population's urbanicity. We provide recommendations on the data selection method and the quantity of past data to use when warm starting AL training in the urban and suburban schools.

Many areas of educational research require the analysis of data that have an inherent sequential or temporal ordering. In certain cases, researchers are specifically interested in the transitions between different states—or events—in these sequences, with the goal being to understand the significance of these transitions; one notable example is the study of affect dynamics, which aims to identify important transitions between affective states. Unfortunately, a recent study has revealed a statistical bias with several metrics used to measure and compare these transitions, possibly causing these metrics to return unexpected and inflated values. This issue then causes extra difficulties when interpreting the results of these transition metrics. Building on this previous work, in this study we look in more detail at the specific mechanisms that are responsible for the bias with these metrics. After giving a theoretical explanation for the issue, we present an alternative procedure that attempts to address the problem with the use of marginal models. We then analyze the effectiveness of this procedure, both by running simulations and by applying it to actual student data. The results indicate that the marginal model procedure seemingly compensates for the bias observed in other transition metrics, thus resulting in more accurate estimates of the significance of transitions between states.

Education research has explored the role of students' affective states in learning, but some evidence suggests that existing models may not fully capture the meaning or frequency of how students transition between different states. In this study we examine the patterns of educationally-relevant affective states within the context of Betty's Brain, an open-ended, computer-based learning system used to teach complex scientific processes. We examine three types of affective transitions based on similarity with the theorized D'Mello and Graesser model, transition between two affective states, and the sustained instances of certain states. We correlate of the frequency of these patterns with learning outcomes and our findings suggest that boredom is a powerful indicator of students' knowledge, but not necessarily indicative of learning. We discuss our findings within the context of both research and theory on affect dynamics and the implications for pedagogical and system design.

Previous studies have demonstrated strong links between students' linguistic knowledge, their affective language patterns and their success in math. Other studies have shown that demographic and click-stream variables in online learning environments are important predictors of math success. This study builds on this research in two ways. First, it combines linguistics and click-stream variables along with demographic information to increase prediction rates for math success. Second, it examines how random variance, as found in repeated participant data, can explain math success beyond linguistic, demographic, and click-stream variables. The findings indicate that linguistic, demographic, and click-stream factors explained about 14% of the variance in math scores. These variables mixed with random factors explained about 44% of the variance.