Knowledge Tracing (KT) models aim to predict student performance from interaction histories in order to support personalised learning. However, many learners generate only limited interaction data, making reliable knowledge-state estimation difficult. Recent contrastive KT methods attempt to address this data sparsity through self-supervised representation learning from augmented versions of individual learner sequences, operating within an intra-learner paradigm, where contrastive signals are derived solely from variations of a single learner's trajectory. We advance prior work by proposing RACL (Retrieval-Augmented Contrastive Learning), a knowledge tracing framework that introduces an inter-learner contrastive paradigm, leveraging the observation that students with similar skill profiles often exhibit comparable learning trajectories. Cross-learner structure therefore provides naturally occurring positive and negative examples that are more pedagogically meaningful than synthetic augmentations. Experiments on four benchmarks demonstrate that RACL achieves +1.2% average AUC improvement over state-of-the-art methods, with 97% performance retention at 20% training data, indicating improved robustness under sparse-learning conditions.

In computer-based learning environments (CBLEs), adopting effective self-regulated learning (SRL) strategies requires sophisticated coordination of multiple SRL processes. While various studies have proposed adaptive SRL scaffolds (i.e. real-time advice on adopting effective SRL processes) and embedded them in CBLEs to facilitate learners' effective use of SRL strategies, two key research gaps remain. First, there is a lack of research on SRL scaffolds that are based on continuous assessment of both learners' SRL processes and learning conditions (e.g., awareness of learning resources) to provide adaptive support. Second, current analytics-based scaffolding mechanisms lack the scalability needed to effectively address multiple learning conditions. Integration of analytics of SRL with generative artificial intelligence (GenAI) can provide scalable scaffolding for real-time SRL processes and evolving conditions. Yet, empirical studies implementing and evaluating effects of this integration remain scarce. To address these limitations, we conducted a randomized control trial, assigning participants to three groups (control, process only, and process with condition groups) to investigate the effects of using GenAI to turn insights from real-time analytics about students' SRL processes and conditions into adaptive scaffolds. The results demonstrate that integrating real-time analytics with GenAI in adaptive SRL scaffolds - addressing both SRL processes and dynamic conditions - promotes more metacognitive learning patterns compared to the control and process-only groups. In addition, the learners showed varying levels of compliance with analytics-based GenAI scaffolds, and this was also reflected in how the learners coordinated their SRL processes, particularly in the performance phase of SRL. This study contributes to the literature by designing, implementing, and evaluating the impact of adaptive scaffolds on learners' SRL processes using real-time analytics with GenAI.

Peer Reviewed

Optimizing short-term production scheduling in open-pit mines using mining fleets is a complex yet essential task with a significant impact on productivity and cost reduction. This study addresses the growing need for intelligent fleet management systems to maximize the utilization of unmanned mining fleets for efficient production scheduling. A multi-objective production scheduling framework was developed, incorporating deep reinforcement learning and the Internet of Things (IoT) for real-time fleet management. The proposed model focuses on minimizing fleet idle time and transportation costs while maximizing total production through parallel control of multiple shovels and mining trucks. IoT-based travel time estimation was integrated to enhance fleet coordination and improve scheduling accuracy. The model was evaluated using both hypothetical (6 loading points, 2 unloading points, 6 shovels, and 40 trucks) and real-world cases (17 loading points, 3 unloading points, 17 shovels, and 117 trucks) from the Sarcheshmeh Copper Mine in Iran. The DQN-IoT model achieved a 19.2% reduction in truck idle time, outperforming Particle Swarm Optimization (PSO) (12.7%) and Non-dominated Sorting Genetic Algorithm II (NSGA-II) (9.2%). Fleet utilization improved by 4.4%, compared to 2.9% (PSO) and 2.1% (NSGA-II). Operational costs were reduced by 5.5%, surpassing the savings of PSO (1.8%) and NSGA-II (1.2%). These results highlight the superiority of the proposed model and the practical benefits of integrating DQN and IoT in real-time fleet scheduling.

Attributed graph clustering is crucial for analyzing complex networks, but integrating heterogeneous structural and attribute information remains a challenging task. Existing methods often struggle to balance these aspects, resulting in suboptimal clustering performance. To address this, we propose DV-ERNMF (Dual-View Entropy Regularized Nonnegative Matrix Factorization), a framework that decomposes the attributed network into two complementary views, structure and attributes, for separate, yet coordinated modeling. In the structural view, we introduce a Symmetric Nonnegative Matrix Factorization (SNMF) model enhanced with entropy-based regularization to yield sharper cluster assignments. For the attribute view, we construct a clustering-specific similarity matrix via subspace learning and apply SNMF to extract a structurally consistent cluster pattern. A new adaptive entropy-based regularizer is applied to enforce consistency between the partitions obtained from both views. The entire model is optimized jointly using a multiplicative update rule with theoretical convergence guarantees. Experimental results on synthetic and real-world networks demonstrate that DV-ERNMF significantly outperforms state-of-the-art methods.

Attributed graph clustering is a fundamental task in network mining, essential for uncovering valuable insights in various applications. However, the heterogeneity of information from structural and attribute spaces poses significant challenges in achieving consistent and meaningful clustering. To address this, we propose Relative Entropy-based Regularized Non-negative Matrix Factorization (RENMF), a novel approach that integrates structural and attribute information through advanced matrix factorization techniques. RENMF employs Symmetric NMF and Projective NMF to extract community membership distributions from the structural and attribute spaces, respectively. By treating these distributions as homogeneous, RENMF preserves distinct, denoised information from both spaces while considering their heterogeneous complementary information. We introduce Relative Entropy (RE) as a novel regularization term to facilitate interaction between these spaces, aiming to maximize consistency between the discovered latent distributions. In this interaction, we leverage the asymmetric property of RE to emphasize attributes as essential complementary information for structural clustering. The RENMF model is solved using a new iterative multiplicative update rule, with convergence theoretically proven. We evaluate RENMF’s effectiveness through extensive experiments on 10 real-world networks, comparing it to 11 state-of-the-art clustering methods. The results demonstrate RENMF’s superiority in ground truth matching and key quality metrics, outperforming existing methods.

Engaging students in the peer review process has been recognized as a valuable educational tool. It not only nurtures a collaborative learning environment where reviewees receive timely and rich feedback but also enhances the reviewer’s critical thinking skills and encourages reflective self-evaluation. However, a common concern arises when students encounter misaligned or conflicting feedback. Not only can such feedback confuse students; but it can also make it difficult for the instructor to rely on the reviews when assigning a score to the work. Addressing this pressing issue, our paper introduces an innovative, AI-assisted approach that is designed to detect and highlight disagreements within formative feedback. We’ve harnessed extensive data from 170 students, analyzing 15,500 instances of peer feedback from a software development course. By utilizing clustering techniques coupled with sophisticated natural language processing (NLP) models, we transform feedback into distinct feature vectors to pinpoint disagreements. The findings from our study underscore the effectiveness of our approach in enhancing text representations to significantly boost the capability of clustering algorithms in discerning disagreements in feedback. These insights bear implications for educators and software development courses, offering a promising route to streamline and refine the peer review process for the betterment of student learning outcomes.

aendees to leave the workshop with a practical understanding of how to engage with learnersourcing.Participants will get hands-on experience with current tools,

Feedback is an effective way to assist students in achieving learning goals. The conceptualisation of feedback is gradually moving from feedback as information to feedback as a learner-centred process. To demonstrate feedback effectiveness, feedback as a learner-centred process should be designed to provide quality feedback content and promote student learning outcomes on the subsequent task. However, it remains unclear how instructors adopt the learner-centred feedback framework for feedback provision in the teaching practice. Thus, our study made use of a comprehensive learner-centred feedback framework to analyse feedback content and identify the characteristics of feedback content among student groups with different performance changes. Specifically, we collected the instructors’ feedback on two consecutive assignments offered by an introductory to data science course at the postgraduate level. On the basis of the first assignment, we used the status of student grade changes (i.e., students whose performance increased and those whose performance did not increase on the second assignment) as the proxy of the student learning outcomes. Then, we engineered and extracted features from the feedback content on the first assignment using a learner-centred feedback framework and further examined the differences of these features between different groups of student learning outcomes. Lastly, we used the features to predict student learning outcomes by using widely-used machine learning models and provided the interpretation of predicted results by using the SHapley Additive exPlanations (SHAP) framework. We found that 1) most features from the feedback content presented significant differences between the groups of student learning outcomes, 2) the gradient boost tree model could effectively predict student learning outcomes, and 3) SHAP could transparently interpret the feature importance on predictions.

The benefits of incorporating scaffolds that promote strategies of self-regulated learning (SRL) to help student learning are widely studied and recognised in the literature. However, the best methods for incorporating them in educational technologies and empirical evidence about which scaffolds are most beneficial to students are still emerging. In this paper, we report our findings from conducting an in-the-field controlled experiment with 797 post-secondary students to evaluate the impact of incorporating scaffolds for promoting SRL strategies in the context of assisting students in creating novel content, also known as learnersourcing. The experiment had five conditions, including a control group that had access to none of the scaffolding strategies for creating content, three groups each having access to one of the scaffolding strategies (planning, externally-facilitated monitoring and self-assessing) and a group with access to all of the aforementioned scaffolds. The results revealed that the addition of the scaffolds for SRL strategies increased the complexity and effort required for creating content, were not positively assessed by learners and led to slight improvements in the quality of the generated content. We discuss the implications of our findings for incorporating SRL strategies in educational technologies.

Peer review has been recognised as a beneficial approach that promotes higher-order learning and provides students with fast and detailed feedback on their work. Still, there are some common concerns and criticisms associated with the use of peer review that limits its adoption. One of the main points of concern is that feedback provided by students may be ineffective and of low quality. Previous works supply three explanations for why students may fail to provide effective feedback: They lack (1) the ability to provide high-quality feedback, (2) the agency to monitor their work or (3) the incentive to invest the required time and effort as they think the quality of the reviews are not reviewed. To help mitigate these shortcomings, this paper presents a complementary peer review approach that integrates training, self-monitoring and AI quality-control assistance to improve peer feedback quality. In particular, informed by higher education research, we built a set of training materials and a self-monitoring checklist for students to consider while writing their reviews. Also, informed by work from natural language processing, we developed quality control functions that automatically assess feedback submitted and prompt students to improve, if necessary. A between-subjects field experiment with 374 participants was conducted to investigate the approach's efficacy. Findings suggest that offering training, self-monitoring, and quality control functionalities to students assigned to the complementary peer review approach resulted in longer feedback that was perceived as more helpful than those who utilised the regular peer review interface. However, this complementary approach does not seem to affect students judgement (leniency or harshness) or confidence in grading. Directions are suggested to further evaluate and refine peer review systems.

The first annual workshop on Learnersourcing: Student-generated Content @ Scale is taking place at Learning @ Scale 2022. This hybrid workshop will expose attendees to the ample opportunities in the learnersourcing space, including instructors, researchers, learning engineers, and many other roles. We believe participants from a wide range of backgrounds and prior knowledge on learnersourcing can both benefit and contribute to this workshop, as learnersourcing draws on work from education, crowdsourcing, learning analytics, data mining, ML/NLP, and many more fields. Additionally, as the learnersourcing process involves many stakeholders (students, instructors, researchers, instructional designers, etc.), multiple viewpoints can help to inform what future student-generated content might be useful, new and better ways to assess the quality of the content and spark potential collaboration efforts between attendees. We ultimately want to show how everyone can make use of learnersourcing and have participants gain hands-on experience using existing tools, create their own learnersourcing activities using them or their own platforms, and take part in discussing the next challenges and opportunities in the learnersourcing space. Our hope is to attract attendees interested in scaling the generation of instructional and assessment content and those interested in the use of online learning platforms.

Increased enrolments in higher education, and the shift to online learning that has been escalated by the recent COVID pandemic, have made it challenging for instructors to assist their students with their learning needs. Contributing to the growing literature on instructor-facing systems, this paper reports on the development of a learning analytics (LA) technique called Student Inspection Facilitator (SIF) that provides an explainable interpretation of students learning behaviour to support instructors with the identification of students in need of attention. Unlike many previous predictive systems that automatically label students, our approach provides explainable recommendations to guide data exploration while still reserving judgement about interpreting student learning to instructors. The insights derived from applying SIF in an introductory Information Systems course with 407 enrolled students suggest that SIF can be utilised independent of the context and can provide a meaningful interpretation of students' learning behaviour towards facilitating proactive support of students.

Abstract The appetite for effective use of information assets has been steadily rising in both public and private sector organisations. However, whether the information is used for social good or commercial gain, there is a growing recognition of the complex socio-technical challenges associated with balancing the diverse demands of regulatory compliance and data privacy, social expectations and ethical use, business process agility and value creation, and scarcity of data science talent. In this vision paper, we present a series of case studies that highlight these interconnected challenges, across a range of application areas. We use the insights from the case studies to introduce Information Resilience, as a scaffold within which the competing requirements of responsible and agile approaches to information use can be positioned. The aim of this paper is to develop and present a manifesto for Information Resilience that can serve as a reference for future research and development in relevant areas of responsible data management.

The Elo rating system has been recognised as an effective method for modelling students and items within adaptive educational systems. A common characteristic across Elo-based learner models is that they are not sensitive to the lag time between two consecutive interactions of a student within the system. Implicitly, this characteristic assumes that students do not learn or forget between two consecutive interactions. However, this assumption seems insufficient in the context of adaptive learning systems where students could have improved their mastery through practising outside of the system or that their mastery may be declined due to forgetting. In this paper, we extend the existing works on the use of rating systems for modelling learners in adaptive educational systems by proposing a new learner model called MV-Glicko that builds on the Glicko rating system. MV-Glicko is sensitive to the lag time between two consecutive interactions of a student within the system and models it as a parameter that captures the confidence of the system in the current inferred rating. We apply MV-Glicko on three public data sets and three data sets obtained from an adaptive learning system and provide evidence that MV-Glicko outperforms other conventional models in estimating students’ knowledge mastery.

Learnersourcing is emerging as a viable learner-centred and pedagogically justified approach for harnessing the creativity and evaluation power of learners as experts-in-training. Despite the increasing adoption of learnersourcing in higher education, understanding students’ behaviour while engaged in learnersourcing and best practices for the design and development of learnersourcing systems are still largely under-researched. This paper offers data-driven reflections and lessons learned from the development and deployment of a learnersourcing adaptive educational system called RiPPLE, which to date, has been used in more than 50-course offerings with over 12,000 students. Our reflections are categorised into examples and best practices on (1) assessing the quality of students’ contributions using accurate, explainable and fair approaches to data analysis, (2) incentivising students to develop high-quality contributions and (3) empowering instructors with actionable and explainable insights to guide student learning. We discuss the implications of these findings and how they may contribute to the growing literature on the development of effective learnersourcing systems and more broadly technological educational solutions that support learner-centred learning at scale.

Engaging students in the creation of learning resources has been demonstrated to have pedagogical benefits and lead to the creation of large repositories of learning resources which can be used to complement student learning in different ways. However, to effectively utilise a learnersourced repository of content, a selection process is needed to separate high-quality from low-quality resources as some of the resources created by students can be ineffective, inappropriate, or incorrect. A common and scalable approach to evaluating the quality of learnersourced content is to use a peer review process where students are asked to assess the quality of resources authored by their peers. However, this method poses the problem of "truth inference" since the judgements of students as experts-in-training cannot wholly be trusted. This paper presents a graph-based approach to propagate the reliability and trust using data from peer and instructor evaluations in order to simultaneously infer the quality of the learnersourced content and the reliability and trustworthiness of users in a live setting. We use empirical data from a learnersourcing system called RiPPLE to evaluate our approach. Results demonstrate that the proposed approach can propagate reliability and utilise the limited availability of instructors in spot-checking to improve the accuracy of the model compared to baseline models and the current model used in the system.

Educational recommender systems (ERSs) aim to adaptively recommend a broad range of personalised resources and activities to students that will most meet their learning needs. Commonly, ERSs operate as a "black box" and give students no insight into the rationale of their choice. Recent contributions from the learning analytics and educational data mining communities have emphasised the importance of transparent, understandable and open learner models (OLMs) that provide insight and enhance learners' understanding of interactions with learning environments. In this paper, we aim to investigate the impact of complementing ERSs with transparent and understandable OLMs that provide justification for their recommendations. We conduct a randomised control trial experiment using an ERS with two interfaces ("Non-Complemented Interface" and "Complemented Interface") to determine the effect of our approach on student engagement and their perception of the effectiveness of the ERS. Overall, our results suggest that complementing an ERS with an OLM can have a positive effect on student engagement and their perception about the effectiveness of the system despite potentially making the system harder to navigate. In some cases, complementing an ERS with an OLM has the negative consequence of decreasing engagement, understandability and sense of fairness.

The value of students developing the capacity to make accurate judgements about the quality of their work and that of others has been widely recognised in higher education literature. However, despite this recognition, little attention has been paid to the development of tools and strategies with the potential both to foster evaluative judgement and to support empirical research into its growth. This paper provides a demonstration of how educational technologies may be used to fill this gap. In particular, we introduce the adaptive learning system RiPPLE and describe how it aims to (1) develop evaluative judgement in large-class settings through suggested strategies from the literature such as the use of rubrics, exemplars and peer review and (2) enable large empirical studies at low cost to determine the effect-size of such strategies. A case study demonstrating how RiPPLE has been used to achieve these goals in a specific context is presented.

The big data revolution is an exciting opportunity for universities, which typically have rich and complex digital data on their learners. It has motivated many universities around the world to invest in the development and implementation of learning analytics dashboards (LADs). These dashboards commonly make use of interactive visualisation widgets to assist educators in understanding and making informed decisions about the learning process. A common operation in analytical dashboards is a 'drill-down', which in an educational setting allows users to explore the behaviour of sub-populations of learners by progressively adding filters. Nevertheless, drill-down challenges exist, which hamper the most effective use of the data, especially by users without a formal background in data analysis. Accordingly, in this paper, we address this problem by proposing an approach that recommends insightful drill-downs to LAD users. We present results from an application of our proposed approach using an existing LAD. A set of insightful drill-down criteria from a course with 875 students are explored and discussed.

Adaptive learning systems (ALSs) aim to provide an efficient, effective and customised learning experience for students by dynamically adapting learning content to suit their individual abilities or preferences. Despite consistent evidence of their effectiveness and success in improving student learning over the past three decades, the actual impact and adoption of ALSs in education remain restricted to mostly research projects. In this paper, we provide a brief overview of reflections and lessons learned from developing and piloting an ALS in a course on relational databases. While our focus has been on adaptive learning, many of the presented lessons are also applicable to the development and adoption of educational tools and technologies in general. Our aim is to provide insight for other instructors, educational researchers and developers that are interested in adopting ALSs or are involved in the implementation of educational tools and technologies.

Educational environments continue to rapidly evolve to address the needs of diverse, growing student populations, while embracing advances in pedagogy and technology. In this changing landscape ensuring the consistency among the assessments for different offerings of a course (within or across terms), providing meaningful feedback about students' achievements, and tracking students' progression over time are all challenging tasks, particularly at scale. Here, a collection of visual Topic Dependency Models (TDMs) is proposed to help address these challenges. It visualises the required topics and their dependencies at a course level (e.g., CS 100) and assessment achievement data at the classroom level (e.g., students in CS 100 Term 1 2016 Section 001) both at one point in time (static) and over time (dynamic). The collection of TDMs share a common, two-weighted graph foundation. An algorithm is presented to create a TDM (static achievement for a cohort). An open-source, proof of concept implementation of the TDMs is under development; the current version is described briefly in terms of its support for visualising existing (historical, test) and synthetic data generated on demand.

Larger student intakes by universities and the rise of education through Massive Open Online Courses has led to less direct contact time with teaching staff for each student. One potential way of addressing this contact deficit is to invite learners to engage in peer learning and peer support; however, without technological support they may be unable to discover suitable peer connections that can enhance their learning experience. Two different research subfields with ties to recommender systems provide partial solutions to this problem. Reciprocal recommender systems provide sophisticated filtering techniques that enable users to connect with one another. To date, however, the main focus of reciprocal recommender systems has been on providing recommendation in online dating sites. Recommender systems for technology enhanced learning have employed and tailored exemplary recommenders towards use in education, with a focus on recommending learning content rather than other users. In this paper, we first discuss the importance of supporting peer learning and the role recommending reciprocal peers can play in educational settings. We then introduce our open-source course-level recommendation platform called RiPPLE that has the capacity to provide reciprocal peer recommendation. The proposed reciprocal peer recommender algorithm is evaluated against key criteria such as scalability, reciprocality, coverage, and quality and shows improvement over a baseline recommender. Primary results indicate that the system can help learners connect with peers based on their knowledge gaps and reciprocal preferences, with designed flexibility to address key limitations of existing algorithms identified in the literature.

Educators continue to face significant challenges in providing high quality, post-secondary instruction in large classes including: motivating and engaging diverse populations (e.g., academic ability and backgrounds, generational expectations); and providing helpful feedback and guidance. Researchers investigate solutions to these kinds of challenges from alternative perspectives, including learning analytics (LA). Here, LA techniques are applied to explore the data collected for a large, flipped introductory programming class to (1) identify groups of students with similar patterns of performance and engagement; and (2) provide them with more meaningful appraisals that are tailored to help them effectively master the learning objectives. Two studies are reported, which apply clustering to analyze the class population, followed by an analysis of a subpopulation with extreme behaviours.

The revolution of social networks and methods of analyzing them have attracted interest in many research fields. Predicting whether a friendship holds in a social network between two individuals or not, link prediction, has been a heavily researched topic in the last decade . In this paper we investigate a related problem, link strength prediction: how to assign ratings or strengths to friendship links. A basic approach would be matrix factorization applied to only friendship ratings. However, the existence of extensive transactions among users may be used for better predictions. We propose a new type of multiple-matrix factorization model for incorporating a transaction matrix. We derive gradient descent update equations for learning latent factors that predict values in the target rating matrix. Multiple-matrix factorization can be seen as a data fusion technique, that combines evidence from different sources. In the social network application, the target matrix contains friendship ratings and the evidence matrices specify transaction intensities between users. To evaluate the model, we introduce data from Cloob, a popular Iranian social network as well as synthetic data.

Recently, there has been an increasing interest in generative models that represent probabilistic patterns over both links and attributes. A common characteristic of relational data is that the value of a predicate often depends on values of the same predicate for related entities. For directed graphical models, such recursive dependencies lead to cycles, which violates the acyclicity constraint of Bayes nets. In this paper we present a new approach to learning directed relational models which utilizes two key concepts: a pseudo likelihood measure that is well defined for recursive dependencies, and the notion of stratification from logic programming. An issue for modelling recursive dependencies with Bayes nets are redundant edges that increase the complexity of learning. We propose a new normal form format that removes the redundancy, and prove that assuming stratification, the normal form constraints involve no loss of modelling power. Empirical evaluation compares our approach to learning recursive dependencies with undirected models (Markov Logic Networks). The Bayes net approach is orders of magnitude faster, and learns more recursive dependencies, which lead to more accurate predictions.

Many machine learning applications that involve relational databases incorporate first-order logic and probability. Markov Logic Networks (MLNs) are a prominent statistical relational model that consist of weighted first order clauses. Many of the current state-of-the-art algorithms for learning MLNs have focused on relatively small datasets with few descriptive attributes, where predicates are mostly binary and the main task is usually prediction of links between entities. This paper addresses what is in a sense a complementary problem: learning the structure of an MLN that models the distribution of discrete descriptive attributes on medium to large datasets, given the links between entities in a relational database. Descriptive attributes are usually nonbinary and can be very informative, but they increase the search space of possible candidate clauses. We present an efficient new algorithm for learning a directed relational model (parametrized Bayes net), which produces an MLN structure via a standard moralization procedure for converting directed models to undirected models. Learning MLN structure in this way is 200-1000 times faster and scores substantially higher in predictive accuracy than benchmark algorithms on three relational databases.

A Bayes net has qualitative and quantitative aspects: The qualitative aspect is its graphical structure that corresponds to correlations among the variables in the Bayes net. The quantitative aspects are the net parameters. This paper develops a hybrid criterion for learning Bayes net structures that is based on both aspects. We combine model selection criteria measuring data fit with correlation information from statistical tests: Given a sample d, search for a structure G that maximizes score(G, d), over the set of structures G that satisfy the dependencies detected in d. We rely on the statistical test only to accept conditional dependencies, not conditional independencies. We show how to adapt local search algorithms to accommodate the observed dependencies. Simulation studies with GES search and the BDeu/BIC scores provide evidence that the additional dependency information leads to Bayes nets that better fit the target model in distribution and structure.