The learning analytics (LA) community has recently reached two important milestones: celebrating the 15th LAK conference and updating the 2011 definition of LA to reflect the 15 years of changes in the discipline. However, despite LA’s growth, little is known about how research topics, funding, and collaboration, as well as the relationships among them, have developed within the community over time. This study addressed this gap by analyzing all 936 full and short papers published at LAK over a 15-year period using unsupervised machine learning, natural language processing, and network analytics. The analysis revealed a stable core of prolific authors alongside high turnover of newcomers, systematic links between funding sources and research directions, and six enduring topical centers that remain globally shared but vary in prominence across countries. These six topical centers, which encompass LA research, are: self-regulated learning, dashboards and theory, social learning, automated feedback, multimodal analytics, and outcome prediction. Our findings highlight key challenges for the future: widening participation, reducing dependency on a narrow set of funders, and ensuring that emerging research trajectories remain responsive to educational practice and societal needs.

In supervised machine learning (SML) research, large training datasets are essential for valid results. However, obtaining primary data in learning analytics (LA) is challenging. Data augmentation can address this by expanding and diversifying data, though its use in LA remains underexplored. This paper systematically compares data augmentation techniques and their impact on prediction performance in a typical LA task: prediction of academic outcomes. Augmentation is demonstrated on four SML models, which we successfully replicated from a previous LAK study based on AUC values. Among 21 augmentation techniques, SMOTE-ENN sampling performed the best, improving the average AUC by 0.01 and approximately halving the training time compared to the baseline models. In addition, we compared 99 combinations of chaining 21 techniques, and found minor, although statistically significant, improvements across models when adding noise to SMOTE-ENN (+0.014). Notably, some augmentation techniques significantly lowered predictive performance or increased performance fluctuation related to random chance. This paper's contribution is twofold. Primarily, our empirical findings show that sampling techniques provide the most statistically reliable performance improvements for LA applications of SML, and are computationally more efficient than deep generation methods with complex hyperparameter settings. Second, the LA community may benefit from validating a recent study through independent replication.

Productively engaging in SRL is challenging for learners since it involves coordinating multiple motivational, affective, cognitive, and metacognitive processes. Researchers have investigated methods to adaptively scaffold learners' productive engagement using SRL processes automatically captured by SRL detectors. However, most previous studies relied solely on the frequency of SRL processes to drive adaptive scaffolds (e.g., feedback, hints), possibly missing the sequential characteristics inherent to self-regulation, a crucial dimension of productive SRL. To address this gap, this study analysed the impact of sequential transitions between multiple SRL processes on learners' performance on a reading-writing task with a hypermedia environment called Flora. A sample of 66 secondary-school learners completed the task and trace data were collected. Grounded in the COPES model of SRL, a rule-based SRL detector was employed to capture SRL processes from collected trace data. We employed a method combining logistic regression with ordered network analysis (ONA) to analyse the transitions between the detected SRL processes. This exploratory study revealed several influential transitions to learners' performance in different temporal learning blocks of self-regulation. The implications suggest the potential of using COPES SRL process transitions to drive adaptive scaffolds to facilitate engagement in productive SRL, benefiting performance outcomes in hypermedia environments.

Abstract Game-based learning (GBL) environments are designed to foster emotional experiences conducive to learning; yet, there are mixed findings regarding their effectiveness. The inconsistent results may stem from challenges in measuring and modeling emotions as multi-dimensional constructs during GBL. Traditional approaches often use one data channel and conventional statistics to study emotions, which limit our understanding of the multi-componential interactions that underlie emotional states during GBL. In this study, we merged non-linear dynamical systems (NLDS) theory with the component process model of emotion to examine interactions and synchrony among two emotion signals during GBL, facial expressions and heart rate variability (HRV), and assessed its relation to knowledge and learning gain. Data were collected from 58 participants (n = 58) at a university in Central Finland while they learned about pathology with a tower defense game called Antidote COVID-19. Results showed a significant improvement in knowledge after GBL. A NLDS technique called cross-wavelet transformation showed there were varying degrees of synchrony between facial expressions and HRV. Neutral expressions showed the highest degree of synchrony with HRV, followed closely by happiness and anger with HRV. However, the synchrony between facial expressions and HRV did not affect knowledge and learning gain. This research contributes to the field by studying emotions as multidimensional systems during GLB.

Programming courses can be challenging for first year university students, especially for those without prior coding experience.Students initially struggle with code syntax, but as more advanced topics are introduced across a semester, the difficulty in learning to program shifts to learning computational thinking (e.g., debugging strategies).This study examined the relationships between students' rate of programming errors and their grades on two exams.Using an online integrated development environment, data were collected from 280 students in a Java programming course.The course had two parts.The first focused on introductory procedural programming and culminated with exam 1, while the second part covered more complex topics and object-oriented programming and ended with exam 2. To measure students' programming abilities, 51095 code snapshots were collected from students while they completed assignments that were autograded based on unit tests.Compiler and runtime errors were extracted from the snapshots, and three measures -Error Count, Error Quotient and Repeated Error Density -were explored to identify the best measure explaining variability in exam grades.Models utilizing Error Quotient outperformed the models using the other two measures, in terms of the explained variability in grades and Bayesian Information Criterion.Compiler errors were significant predictors of exam 1 grades but not exam 2 grades; only runtime errors significantly predicted exam 2 grades.The findings indicate that leveraging Error Quotient with multiple error types (compiler and runtime) may be a better measure of students' introductory programming abilities, though still not explaining most of the observed variability.

A student’s ability to accurately evaluate the quality of their work holds significant implications for their self-regulated learning and problem-solving proficiency in introductory programming. A widespread cognitive bias that frequently impedes accurate self- assessment is overconfidence, which often stems from a misjudgment of contextual and task-related cues, including students’ judgment of their peers’ competencies. Little research has explored the role of overconfidence on novice programmers’ ability to accurately monitor their own work in comparison to their peers’ work and its impact on performance in introductory programming courses. The present study examined whether novice programmers exhibited a common cognitive bias called the "hard-easy effect", where students believe their work is better than their peers on easier tasks (overplace) but worse than their peers on harder tasks (underplace). Results showed a reversal of the hard-easy effect, where novices tended to overplace themselves on harder tasks, yet underplace themselves on easier ones. Remarkably, underplacers performed better on an exam compared to overplacers. These findings advance our understanding of relationships between the hard-easy effect, monitoring accuracy across multiple tasks, and grades within introductory programming. Implications of this study can be used to guide instructional decision making and design to improve novices’ metacognitive awareness and performance in introductory programming courses.

Numerous studies aim to enhance learning in digital environments through emotionally-sensitive interventions. The D’Mello and Graesser (2012) model of affect dynamics hypothesizes that when a learner encounters confusion, the degree to which it is prolonged (and transitions into frustration) or resolved, significantly affects their learning outcomes in digital environments. However, studies yield inconclusive results regarding relations between confusion, frustration, and learning. More research is needed to explore how confusion and frustration manifest during learning and its relation to outcomes. We go beyond past work looking at the rate, duration, and transitions of confusion and frustration by treating these affective states as non-linear dynamical systems consisting of expressive and behavioral components. We examined the frequency and recurrence of facial expressions associated with basic emotions (as automatically labeled by AffDex, a standard tool for analyzing emotions with video data) during confused and frustrated states (as automatically labeled with BROMP-based detectors applied to students’ interaction data). We compare these co-occurring patterns to learning outcomes (pre-tests, post-tests, and learning gains) within a digital learning environment, Betty’s Brain. Results showed that the frequency and recurrence rate of basic emotions expressed during confusion and frustration are complex and remain incompletely understood. Specifically, we show that confusion and frustration have different relationships with learning outcomes, depending on which basic emotion expressions they co-occur with. Implications of this study open avenues for better understanding these emotions as complex and non-linear dynamical systems, in the long-term enabling personalized feedback and emotional support within digital learning environments that enhance learning outcomes.

When students procrastinate on programming assignments, it can hinder the quality of their code and negatively impact their grades. In contrast, when students actively delay working on assignments to prepare to code (e.g., reading or seeking help), it can be an effective self-regulated learning (SRL) strategy beneficial to programming performance. However, distinguishing active delay from procrastination is methodologically challenging. To address this, we tracked what students did when they behaviorally delayed starting an assignment. Most students prepared to code by using multiple course resources across programming assignments. We found that many students delayed starting to code by seeking help in the Q&A platform, and this was beneficial to the quality of their code. Also, some pre-coding activities were related to behavioral delay in starting to code, but benefitted students' grades, and thus may indicate active delay, but not all pre-coding activities were beneficial. By considering pre-coding activities, we gain a comprehensive view of students' approach to coding in CS education.

The study investigates the impact of cognitive biases on middle-school students' affective experiences while learning about math in a game-based learning environment (GBLE). The study focused on students' confrustion, an affect construct that unifies the several manifestations of confusion and frustration. We studied confrustion in the context of students self-explaining erroneous examples, where they had to find and fix common errors in given math problems and self-explain their problem-solving processes either with or without scaffolding. Text replays were utilized to examine student interactions during game-based learning and identify behaviors that emerged in response to cognitive biases and affect and its impact on learning and performance outcomes. The results revealed that students who demonstrated more pseudo-confidence in their self-explanations had higher self-reported self-efficacy, but were more likely to submit incongruent responses, exhibit confrustion, make errors, and take longer to finish the game. Overall, the findings show that students were vulnerable to cognitive biases and did not always respond in ways that accurately reflected their approach to solving math problems. The insights into how students approach and learn from math games inform the design and implementation of GBLEs by addressing cognitive biases.

Self-regulated learning (SRL) is important for computer science education. Yet, students often do not have SRL skills to benefit their learning. In this study, we examined 187 (n=187) students’ SRL behaviors while they built programs with an automated feedback tool. Anchored in Winne and Hadwin’s (1998) COPES model of SRL, our results showed that novices used more operators to debug compiler errors, while more experienced programmers used more operators to debug non-compiler errors. Finally, a random forest classifier showed that prior knowledge was the most important COPES feature predicting learning gain, followed closely by the student’s perceived programming ability, use of evaluations with the automated feedback tool, and operators used to debug non-compiler errors on failed programs.

Computing education researchers often study the impact of online help-seeking behaviors that occur across multiple online resources in isolation. Such separation fails to capture the interconnected nature of online help-seeking behaviors that occur across multiple online resources and its affect on course grades. This is particularly important for programming education, which arguably has more online resources to seek help from other people (e.g., computer-mediated conversations) than other majors. Using data from an introductory programming course (CS1) at a large US university, we found that students (n=301) sought help in multiple computer-mediated conversations, both Q&A forum and online office hours (OHQ), differently. Results showed the more prior knowledge about programming students had, the more they sought help in the Q&A compared to students with less prior knowledge. In general, higher-performing students sought help online in the Q&A more than the lower-performing groups on all the homework assignments, but not for the OHQ. By better understanding how students seek help online across multiple modalities of computer-mediated conversations and the relationship between help-seeking and grades, we can re-design online resources that best support all students in introductory programming courses at scale.

Inability to regulate affective states can impact one's capacity to engage in higher-order thinking like scientific reasoning with game-based learning environments. Many efforts have been made to build affect-aware systems to mitigate the potentially detrimental effects of negative affect. Yet, gaps in research exist since accurately capturing and modeling affect as a state that changes dynamically over time is methodologically and analytically challenging. In this paper, we calculated multilevel mixed effects growth models to assess whether seventy-eight participants’ (n= 78) time engaging in scientific reasoning (via logfiles and eye gaze) were related to time facially expressing confused, frustrated, and neutral states (via facial recognition software) during game-based learning with Crystal Island. The fitted model estimated significant positive relations between the time learners facially expressed confusion, frustration, and neutral states and time engaging in scientific-reasoning actions. The time individual learners facially expressed frustrated, confused, and neutral states explained a significant amount of variation in time engaging in scientific reasoning. Our finding emphasize that individual differences and agency may play a important role on relations between affective states, their dynamics, and higher-order cognition during game-based learning. Designing affect-aware game-based learning environments that track the dynamics within individual learners’ affective states may best support cognition.

Significant efforts are currently being made to design affect-aware systems to classify, monitor, and scaffold emotional experiences across a range of settings. However, most investigations are limited in their view of emotions due to less sophisticated methodologies and analytical techniques. To address these issues, we captured 174 undergraduates’ emotions over time and defined them by multiple dimensions: (1) temporality, (2) valence, and (3) activation during learning with an intelligent tutoring system called MetaTutor. Latent growth models revealed the stability of negative activating emotions over time was related to performance, and changes in negative deactivating emotions were related to time engaging in cognitive strategies. Finally, a random forest classifier revealed high accuracy in predicting high (top 30%) and low performance groups (bottom 30%) using pre-test scores, changes in negative deactivating emotions, and time engaging in cognitive strategies. These findings have important implications for designing affect-aware systems that can potentially leverage emotion interventions based on if, when, and how an emotion changed (or remained stable) to optimize cognition and performance with emerging technologies.

Reflection plays a critical role in learning by encouraging students to contemplate their knowledge and previous learning experiences to inform their future actions and higher-order thinking, such as reasoning and problem solving. Reflection is particularly important in inquiry-driven learning scenarios where students have the freedom to set goals and regulate their own learning. However, despite the importance of reflection in learning, there are significant theoretical, methodological, and analytical challenges posed by measuring, modeling, and supporting reflection. This paper presents results from a classroom study to investigate middle-school students’ reflection during inquiry-driven learning with Crystal Island, a game-based learning environment for middle-school microbiology. To collect evidence of reflection during game-based learning, we used embedded reflection prompts to elicit written reflections during students’ interactions with Crystal Island. Results from analysis of data from 105 students highlight relationships between features of students’ reflections and learning outcomes related to both science content knowledge and problem solving. We consider implications for building adaptive support in game-based learning environments to foster deep reflection and enhance learning, and we identify key features in students’ problem-solving actions and reflections that are predictive of reflection depth. These findings present a foundation for providing adaptive support for reflection during game-based learning.