Learning analytics can guide human tutors to efficiently address motivational barriers to learning that AI systems struggle to support. Students become more engaged when they receive human attention. However, what occurs during short interventions, and when are they most effective? We align student–tutor dialogue transcripts with MATHia tutoring system log data to study brief human-tutor interactions on Zoom drawn from 2,075 hours of 191 middle school students’ classroom math practice. Mixed-effect models reveal that engagement, measured as successful solution steps per minute, is higher during a human-tutor visit and remains elevated afterward. Visit length exhibits diminishing returns: engagement rises during and shortly after visits, irrespective of visit length. Timing also matters: later visits yield larger immediate lifts than earlier ones, though an early visit remains important to counteract engagement decline. We create analytics that identify which tutor-student dialogues raise engagement the most. Qualitative analysis reveals that interactions with concrete, stepwise scaffolding with explicit work organization elevate engagement most strongly. We discuss implications for resource-constrained tutoring, prioritizing several brief, well-timed check-ins by a human tutor while ensuring at least one early contact. Our analytics can guide the prioritization of students for support and surface effective tutor moves in real-time.

The expansion of large-scale online education platforms has yielded vast amounts of student interaction data for knowledge tracing (KT). KT models estimate students’ concept mastery from interaction data, but the models’ performance is sensitive to input data quality. Gaming behaviors, such as excessive hint use, may misrepresent students’ knowledge and undermine model reliability. However, systematic investigations of how different types of gaming behaviors affect KT remain scarce, and existing studies rely on costly manual analysis that does not capture behavioral diversity. In this study, we conceptualize gaming behaviors as a form of data poisoning, defined as the deliberate submission of incorrect or misleading interaction data to corrupt a model’s learning process. We design Data Poisoning Attacks (DPA) to simulate diverse gaming patterns and systematically evaluate their impact on KT model performance. Moreover, drawing on advances in DPA detection, we explore unsupervised approaches to enhance the generalizability of gaming behavior detection. We find that KT models performance tend to decrease especially for random guess behaviors. Our findings provide insights into the vulnerabilities of KT models and highlight the potential of adversarial methods for improving the robustness of learning analytics systems.

Educational early-warning systems increasingly operate across campuses and platforms, but combining learner records into a single repository is often restricted by privacy and governance constraints. We study whether federated learning---a training approach in which institutions keep records local and share only model updates---can serve as a practical alternative to centralised training for predicting which students are at risk of poor outcomes, without sacrificing the triage decisions the system makes. Using three public datasets spanning roughly 700 to 120,000 learners, we compare centralised training and federated training across simulated institutional clients under matched preprocessing, model family, and a controlled training schedule. We evaluate predictive accuracy, calibration, and the extent to which each approach identifies students most likely to experience the target poor outcome under limited advising capacity, and then stress-test privacy and robustness to malicious participants. On the medium and large datasets, FedAvg closely matches Central on aggregate ranking quality and shortlist precision, although individual scores and selected students can differ. Under matched privacy strength, utility remains similar, but privacy loss accumulates unevenly across institutions, and a defensive aggregation rule reduces---but does not eliminate---the effect of malicious participants. These results position federated learning as a viable collaboration pattern for learning-at-scale deployments when raw data cannot be shared, provided calibration, privacy, and integrity are evaluated explicitly rather than assumed.

Foundation models for tabular data, such as the Tabular Prior-data Fitted Network (TabPFN), are pre-trained on a massive number of synthetic datasets generated by structural causal models (SCM). They leverage in-context learning to offer high predictive accuracy in real-world tasks. However, the fairness properties of these foundational models, which incorporate ideas from causal reasoning during pre-training, remain underexplored. In this work, we conduct a comprehensive empirical evaluation of TabPFN and its fine-tuned variants, assessing predictive performance, fairness, and robustness across varying dataset sizes and distributional shifts. Our results reveal that while TabPFN achieves stronger predictive accuracy compared to baselines and exhibits robustness to spurious correlations, improvements in fairness are moderate and inconsistent, particularly under missing-not-at-random (MNAR) covariate shifts. These findings suggest that the causal pre-training in TabPFN is helpful but insufficient for algorithmic fairness, highlighting implications for deploying TabPFN (and similar) models in practice and the need for further fairness interventions.

A bstract--Institutions are increasingly interested in incorporating remote labs into higher education settings. In this paper, we present our high-fidelity design of an instructor-facing dashboard grounded for remote lab settings, developed using the LATUX workflow. The dashboard leverages multidimensional data sources to track student activity. The first data source focuses on individual-based learning, where students explore recorded lab experiments with detailed context. The second supports collaborative learning, facilitated through touch-screen interfaces. Our dashboard design incorporates visualizations such as heatmaps and activities to help instructors make data-driven decisions. This work offers insights into our development process and key factors for enhancing remote lab instruction.

Concerns about the lack of impact of learning analytics (LA) research has been part of the evolution of the field since its emergence as a research focus and practice in 2011. The preponderance of small-scale and exploratory nature of much of LA research are well-documented as contributing factors to the lack of generalisability, transferability, replicability and scalability. Through an analysis of 144 full research papers published in the conference proceedings of the Learning Analytics & Knowledge (LAK) Conference '22, 23 and 24, this paper provides an overview of the extent and contours of the lack of generalisability in LA research and pointers for making LA research more generalisable. The inductive and deductive analysis of the recent three LAK conferences provide evidence that a significant percentage (46%) of the corpus papers do not refer at all to generalisability or transferability, while few papers report on the scalability of their research findings. While the crisis of replicability/reproducibility is a wider concern in the broader context of research, considering and reporting on generalisability and transferability is integral to the scientific rigour. We conclude our paper with a range of pointers for addressing the lack of generalisability in LA research including, but not limited to expanding data, methodological adaptation and the potential of open science.

This paper addresses the challenge of balancing learner data privacy with the use of data in learning analytics (LA) by proposing a novel framework by applying Differential Privacy (DP). The need for more robust privacy protection keeps increasing, driven by evolving legal regulations and heightened privacy concerns, as well as traditional anonymization methods being insufficient for the complexities of educational data. To address this, we introduce the first DP framework specifically designed for LA and provide practical guidance for its implementation. We demonstrate the use of this framework through a LA usage scenario and validate DP in safeguarding data privacy against potential attacks through an experiment on a well-known LA dataset. Additionally, we explore the trade-offs between data privacy and utility across various DP settings. Our work contributes to the field of LA by offering a practical DP framework that can support researchers and practitioners in adopting DP in their works.

The recent advances in educational technology enabled the development of solutions that collect and analyse data from learning scenarios to inform the decision-making processes. Research fields like Learning Analytics (LA) and Artificial Intelligence (AI) aim at supporting teaching and learning by using such solutions. However, their adoption in authentic settings is still limited, among other reasons, derived from ignoring the stakeholders' needs, a lack of pedagogical contextualisation, and a low trust in new technologies. Thus, the research fields of Human-Centered LA (HCLA) and Human-Centered AI (HCAI) recently emerged, aiming to understand the active involvement of stakeholders in the creation of such proposals. This paper presents a systematic literature review of 47 empirical research studies on the topic. The results show that more than two-thirds of the papers involve stakeholders in the design of the solutions, while fewer papers involved them during the ideation and prototyping, and the majority do not report any evaluation. Interestingly, while multiple techniques were used to collect data (mainly interviews, focus groups and workshops), few papers explicitly mentioned the adoption of existing HC design guidelines. Further evidence is needed to show the real impact of HCLA/HCAI approaches (e.g., in terms of user satisfaction and adoption).

In the context of Bayesian inversion for scientific and engineering modeling, Markov chain Monte Carlo (MCMC) sampling strategies have become the benchmark due to their flexibility and robustness in dealing with arbitrary posterior probability density functions (PDFs). However, these algorithms have been shown to be inefficient when sampling from high-dimensional posterior distributions or exhibit multimodality and/or strong parameter correlations. In such contexts, transitional MCMC (TMCMC) provides a more efficient alternative. Despite the recent applicability for Bayesian updating and model selection across a variety of disciplines, TMCMC may require a prohibitive number of tempering stages when the prior pdf is significantly different from the target posterior. Furthermore, the need to start with an initial set of samples from the prior distribution may present a challenge when dealing with implicit priors, e.g., based on feasible regions. Finally, TMCMC cannot be used for inverse problems with improper prior PDFs that represent a lack of prior knowledge on all or a subset of parameters. A generalization of TMCMC is proposed that alleviates such challenges and limitations toward providing a more robust and efficient tempering sampling strategy. We present convergence analysis, proving that the distance between the intermediate distributions and the target posterior distribution monotonically decreases as the algorithm proceeds. We also demonstrate the advantages of the proposed generalization through a series of test problems and an engineering application in the oil and gas industry.

As online education becomes more widely available, the amount of data available and accessible has exploded. Such a wealth of educational data provides ample opportunities for the fields of learning analytics and educational data mining to expand and yield numerous benefits, such as identifying at-risk students, providing personalized feedback, and providing actionable insights. Machine learning and deep learning techniques are commonly used for these and other purposes. However, simply applying these technologies is not enough without allowing humans to understand them. Because only those who understand the logic behind the technology can cultivate trust and make the technology work to its maximum effectiveness. To address the gap, this paper focuses specifically on a case study of explainable machine learning techniques for predicting student performance in online courses, and its contribution is twofold. First, we show how the use of explainable AI techniques can inform model diagnosis and direction forward in situations where performance is less than ideal. Secondly, we show how different types of explainable AI techniques can improve transparency and trust in educational scenarios, allowing stakeholders to benefit from explainable AI.

While learning analytics dashboards (LADs) are the most common form of LA intervention, there is limited evidence regarding their impact on students’ learning outcomes. This systematic review synthesizes the findings of 38 research studies to investigate the impact of LADs on students' learning outcomes, encompassing achievement, participation, motivation, and attitudes. As we currently stand, there is no evidence to support the conclusion that LADs have lived up to the promise of improving academic achievement. Most studies reported negligible or small effects, with limited evidence from well-powered controlled experiments. Many studies merely compared users and non-users of LADs, confounding the dashboard effect with student engagement levels. Similarly, the impact of LADs on motivation and attitudes appeared modest, with only a few exceptions demonstrating significant effects. Small sample sizes in these studies highlight the need for larger-scale investigations to validate these findings. Notably, LADs showed a relatively substantial impact on student participation. Several studies reported medium to large effect sizes, suggesting that LADs can promote engagement and interaction in online learning environments. However, methodological shortcomings, such as reliance on traditional evaluation methods, self-selection bias, the assumption that access equates to usage, and a lack of standardized assessment tools, emerged as recurring issues. To advance the research line for LADs, researchers should use rigorous assessment methods and establish clear standards for evaluating learning constructs. Such efforts will advance our understanding of the potential of LADs to enhance learning outcomes and provide valuable insights for educators and researchers alike.

Privacy poses a significant obstacle to the progress of learning analytics (LA), presenting challenges like inadequate anonymization and data misuse that current solutions struggle to address. Synthetic data emerges as a potential remedy, offering robust privacy protection. However, prior LA research on synthetic data lacks thorough evaluation, essential for assessing the delicate balance between privacy and data utility. Synthetic data must not only enhance privacy but also remain practical for data analytics. Moreover, diverse LA scenarios come with varying privacy and utility needs, making the selection of an appropriate synthetic data approach a pressing challenge. To address these gaps, we propose a comprehensive evaluation of synthetic data, which encompasses three dimensions of synthetic data quality, namely resemblance, utility, and privacy. We apply this evaluation to three distinct LA datasets, using three different synthetic data generation methods. Our results show that synthetic data can maintain similar utility (i.e., predictive performance) as real data, while preserving privacy. Furthermore, considering different privacy and data utility requirements in different LA scenarios, we make customized recommendations for synthetic data generation. This paper not only presents a comprehensive evaluation of synthetic data but also illustrates its potential in mitigating privacy concerns within the field of LA, thus contributing to a wider application of synthetic data in LA and promoting a better practice for open science.

Multimodal Learning Analytics (MMLA) has been applied to collaborative learning, often to estimate collaboration quality with the use of multimodal data, which often have uneven time scales. The difference in time scales is usually handled by dividing and aggregating data using a fixed-size time window. So far, the current MMLA research lacks a systematic exploration of whether and how much window size affects the generalizability of collaboration quality estimation models. In this paper, we investigate the impact of different window sizes (e.g., 30 seconds, 60s, 90s, 120s, 180s, 240s) on the generalizability of classification models for collaboration quality and its underlying dimensions (e.g., argumentation). Our results from an MMLA study involving the use of audio and log data showed that a 60 seconds window size enabled the development of more generalizable models for collaboration quality (AUC 61%) and argumentation (AUC 64%). In contrast, for modeling dimensions focusing on coordination, interpersonal relationship, and joint information processing, a window size of 180 seconds led to better performance in terms of across-context generalizability (on average from 56% AUC to 63% AUC). These findings have implications for the eventual application of MMLA in authentic practice.

While learning analytics frameworks precede the official launch of learning analytics in 2011, there has been a proliferation of learning analytics frameworks since. This systematic review of learning analytics frameworks between 2011 and 2021 in three databases resulted in an initial corpus of 268 articles and conference proceeding papers based on the occurrence of “learning analytics” and “framework” in titles, keywords and abstracts. The final corpus of 46 frameworks were analysed using a coding scheme derived from purposefully selected learning analytics frameworks. The results found that learning analytics frameworks share a number of elements and characteristics such as source, development and application focus, a form of representation, data sources and types, focus and context. Less than half of the frameworks consider student data privacy and ethics. Finally, while design and process elements of these frameworks may be transferable and scalable to other contexts, users in different contexts will be best-placed to determine their transferability/scalability.

Twitter is a very popular microblogging platform that has been actively used by scientific communities to exchange scientific information and to promote scholarly discussions. The present study aimed to leverage the tweet data to provide valuable insights into the development of the learning analytics field since its initial days. Descriptive analysis, geocoding analysis, and topic modeling were performed on over 1.6 million tweets related to learning analytics posted between 2010-2021. The descriptive analysis reveals an increasing popularity of the field on the Twittersphere in terms of number of users, twitter posts, and hashtags emergence. The topic modeling analysis uncovers new insights of the major topics in the field of learning analytics. Emergent themes in the field were identified, and the increasing (e.g., Artificial Intelligence) and decreasing (e.g., Education) trends were shared. Finally, the geocoding analysis indicates an increasing participation in the field from more diverse countries all around the world. Further findings are discussed in the paper.

Self-regulated learning (SRL) can predict academic performance. Yet, it is difficult for learners. The ability to self-regulate learning becomes even more important in emerging online learning settings. To support learners in developing their SRL, learning analytics (LA), which can improve learning practice by transforming the ways we support learning, is critical. This scoping review is based on the analysis of 54 papers on LA empirical research for SRL in online learning contexts published between 2011 and 2019. The research question is: What is the current state of the applications of learning analytics to measure and support students' SRL in online learning environments? The focus is on SRL phases, methods, forms of SRL support, evidence for LA and types of online learning settings. Zimmerman's model (2002) was used to examine SRL phases. The evidence about LA was examined in relation to four propositions: whether LA i) improve learning outcomes, ii) improve learning support and teaching, iii) are deployed widely, and iv) used ethically. Results showed most studies focused on SRL parts from the forethought and performance phase but much less focus on reflection. We found little evidence for LA that showed i) improvements in learning outcomes (20%), ii) improvements in learning support and teaching (22%). LA was also found iii) not used widely and iv) few studies (15%) approached research ethically. Overall, the findings show LA research was conducted mainly to measure rather than to support SRL. Thus, there is a critical need to exploit the LA support mechanisms further in order to ultimately use them to foster student SRL in online learning environments.

Evidence suggests that individuals are often willing to exchange personal data for (real or perceived) benefits. Such an exchange may be impacted by their trust in a particular context and their (real or perceived) control over their data.

A Massive Open Online Course (MOOC) is a type of online learning environment that has the potential to increase students' access to education. However, the low completion rates in MOOCs suggest that student engagement and progression in the courses are problematic. Following the increasing adoption of gamification in education, it is possible that gamification can also be effectively adopted in MOOCs to enhance students' motivation and increase completion rates. Yet at present, the extent to which gamification has been examined in MOOCs is not known. Considering the myriad gamification elements that can be adopted in MOOCs (e.g., leaderboards and digital badges), this theoretical research study reviews scholarly publications examining gamification of MOOCs. The main purpose is to provide an overview of studies on gamification in MOOCs, types of research studies, theories applied, gamification elements implemented, methods of implementation, the overall impact of gamification in MOOCs, and the challenges faced by researchers and practitioners when implementing gamification in MOOCs. The results of the literature study indicate that research on gamification in MOOCs is in its early stages. While there are only a handful of empirical research studies, results of the experiments generally showed a positive relation between gamification and student motivation and engagement. It is concluded that there is a need for further studies using educational theories to account for the effects of employing gamification in MOOCs.

While many strategies for protecting personal privacy have relied on regulatory frameworks, consent and anonymizing data, such approaches are not always effective. Frameworks and Terms and Conditions often lag user behaviour and advances in technology and software; consent can be provisional and fragile; and the anonymization of data may impede personalized learning. This paper reports on a dialogical multi-case study methodology of four Massive Open Online Course (MOOC) providers from different geopolitical and regulatory contexts. It explores how the providers (1) define 'personal data' and whether they acknowledge a category of 'special' or 'sensitive' data; (2) address the issue and scope of student consent (and define that scope); and (3) use student data in order to inform pedagogy and/or adapt the learning experience to personalise the context or to increase student retention and success rates.

We present a mobile game app (EUR Game) that has been designed to complement teaching and learning in higher education. The mobile game app can be used by teachers to gauge how well students are meeting the learning objectives. Teachers can use the information to provide 'just-in-time' support and adapt their lessons accordingly. For the students, the game app is a study tool that can be used to test their own understanding and monitor their study progress. This, in turn, supports students' self-regulated learning. Gamification elements are also included in the game app to enhance the learning experience. During the demonstration, participants will experience the features of the game app and be engaged in an interactive session to explore the possible ways to use the mobile game app to support teaching and learning.