Simulation-based learning is central to healthcare education, yet its effectiveness depends on high-quality debriefing. Traditional debriefs often overlook detailed team dialogue dynamics. Advances in large language models (LLMs) open new possibilities for learning analytics (LA) by automatically coding and visualising teamwork behaviours from dialogue data. This study investigates the effectiveness of different prompting strategies for LLM-based coding, comparing their performance and environmental impacts (CO2e) to identify approaches suitable for transfer into professional practice. Building on these results, we evaluate the generalisability of the optimised model from university student simulations to in-situ, hospital settings, and explore how healthcare professionals perceive the interpretability, usefulness, and trustworthiness of LLM-driven learning analytics in professional learning debriefs. Findings illustrate that responsible uses of AI can help extend LA beyond a controlled university environment into an authentic, in-hospital healthcare context, offering potentially scalable and sustainable support for reflective practice and professional development.

Assessment of medication history-taking has traditionally relied on human observation, limiting scalability and detailed performance data. While Generative AI (GenAI) platforms enable extensive data collection and learning analytics provide powerful methods for analyzing educational traces, these approaches remain largely underexplored in pharmacy clinical training. This study addresses this gap by applying learning analytics to understand how students develop clinical communication competencies with GenAI-powered virtual patients—a crucial endeavor given the diversity of student cohorts, varying language backgrounds, and the limited opportunities for individualized feedback in traditional training settings. We analyzed 323 students’ interaction logs across Australian and Malaysian institutions, comprising 50,871 coded utterances from 1,487 student-GenAI dialogues. Combining Epistemic Network Analysis to model inquiry co-occurrences with Sequential Pattern Mining to capture temporal sequences, we found that high performers demonstrated strategic deployment of information recognition behaviors. Specifically, high performers centered inquiry on recognizing clinically relevant information, integrating rapport-building and structural organization, while low performers remained in routine question-verification loops. Demographic factors including first-language background, prior pharmacy work experience, and institutional context, also shaped distinct inquiry patterns. These findings reveal inquiry patterns that may indicate clinical reasoning development in GenAI-assisted contexts, providing methodological insights for health professions education assessment and informing adaptive GenAI system design that supports diverse learning pathways.

The knowledge tracing (KT) model based on deep learning has been proven to be superior to the traditional knowledge tracing model, eliminating the need for artificial engineering features. However, there are still problems, such as insufficient interpretability of the learning and answering processes. To address these issues, we propose a new approach in knowledge tracing with attention-based embedding and forgetting curve integration, namely KVFKT. Firstly, the embedding representation module is responsible for embedding the questions and computing the attention vector of knowledge concepts (KCs) when students answer questions and when answer time stamps are collected. Secondly, the forgetting quantification module performs the pre-prediction update of the student’s knowledge state matrix. This quantification involves calculating the interval time and associated forgetting rate of relevant KCs, following the forgetting curve. Thirdly, the answer prediction module generates responses based on students’ knowledge status, guess coefficient, and question difficulty. Finally, the knowledge status update module further refines the students’ knowledge status according to their answers to the questions and the characteristics of those questions. In the experiment, four real-world datasets are used to test the model. Experimental results show that KVFKT better traces students’ knowledge state and outperforms state-of-the-art models.

High staff turnover in higher education often burdens teachers with laborious handovers of teaching tasks every semester. To boost teachers' workflow efficiency, we present an innovative knowledge management tool that allows experienced teachers to seamlessly capture task steps (i.e., denoted as ShareFlow) that can be subsequently recommended to novices via proactive push, all happening during teachers' natural workflow to minimize disruptions. We conducted a controlled experiment with 30 participants and compared our tool against a state-of-the-art baseline knowledge management system powered by a large language model (Claude 3 Haiku). We found that our knowledge management tool reduced task completion time and improved task quality (with statistical significance). Feedback from the participants also indicated the high usability of our tool, suggesting its strong potential for practical adoption for improving teacher workflows.

Temporal patterns play a significant role in understanding dynamic changes in Self-regulated Learning (SRL) engagement over time. Several previous studies have proposed approaches for automated detection of SRL strategies through analysis of temporal patterns. However, these approaches are mostly focused on the analysis of patterns in sequential ordering of SRL processes. This offers a useful yet limited temporal perspective to SRL. As noted in the literature, temporality of SRL has two dimensions - passage of time and ordering of events. To address this gap, this paper specifically proposes a time series approach that can automatically detect SRL strategies by accounting for both dimensions of temporality. Our approach also explores when specific processes occur and how learners engage metacognitively or cognitively with learning tasks. In particular, this study investigated SRL engagement as students composed essays using multiple sources within a 120-minute time frame. The results indicated that five distinct strategies with varying levels of engagement were detected. The correlation between these identified strategies and students' scores was not statistically significant; however, further exploration revealed that students who adopted a specific strategy could outperform other groups based on obtained scores. We also noticed additional factors that had a positive effect on learners' performance.

Collaborative problem-solving (CPS) is defined as an inherently sociocognitive phenomena. Despite this, extant learning analytic techniques tend to focus on either the social or cognitive aspects without explicitly considering their interaction. Prior work developed Neural Epistemic Network Analysis (NENA), which used a combination of deep learning methods to simultaneously model the social and cognitive aspects of CPS; however, the method had several limitations. The refined version of NENA presented here addresses these limitations by (a) introducing a simplified autoencoder deep learning architecture; (b) using a combination of social and epistemic networks as input to preserve interpretability in terms of social and cognitive factors; and (c) introducing an isometry loss function to ensure downstream statistical tests are meaningful. We found that the refined version of NENA is able to achieve high performance on criteria we would expect from a network analytic technique in the context of learning analytics: interpretability, goodness of fit, orthogonality and isometry; and discriminatory power. We also demonstrated that this method was comparable in performance to a more traditional learning analytic technique, Epistemic Network Analysis (ENA), while providing information that ENA did not. The results suggest that NENA could be a useful method for exploring the cognitive interactions of a given individual's social network and thus the influences their network exerts upon them.

Students are increasingly relying on Generative AI (GAI) to support their writing - a key pedagogical practice in education. In GAI-assisted writing, students can delegate core cognitive tasks (e.g., generating ideas and turning them into sentences) to GAI while still producing high-quality essays. This creates new challenges for teachers in assessing and supporting student learning, as they often lack insight into whether students are engaging in meaningful cognitive processes during writing or how much of the essay's quality can be attributed to those processes. This study aimed to help teachers better assess and support student learning in GAI-assisted writing by examining how different writing behaviors, especially those indicative of meaningful learning versus those that are not, impact essay quality. Using a dataset of 1,445 GAI-assisted writing sessions, we applied the cutting-edge method, X-Learner, to quantify the causal impact of three GAI-assisted writing behavioral patterns (i.e., seeking suggestions but not accepting them, seeking suggestions and accepting them as they are, and seeking suggestions and accepting them with modification) on four measures of essay quality (i.e., lexical sophistication, syntactic complexity, text cohesion, and linguistic bias). Our analysis showed that writers who frequently modified GAI-generated text - suggesting active engagement in higher-order cognitive processes - consistently improved the quality of their essays in terms of lexical sophistication, syntactic complexity, and text cohesion. In contrast, those who often accepted GAI-generated text without changes, primarily engaging in lower-order processes, saw a decrease in essay quality. Additionally, while human writers tend to introduce linguistic bias when writing independently, incorporating GAI-generated text - even without modification - can help mitigate this bias.

Conventional classroom instruction often struggles to effectively convey cultural heritage due to constraints in spatial and temporal dimensions, limiting students' ability to fully engage with and appreciate historical content. In contrast, virtual reality (VR) technology offers a human-centered, immersive way to present cultural heritage, creating a dynamic digital experience particularly beneficial when physical access to heritage sites is unavailable. This study investigates whether VR-based learning can enhance students' performance in cultural education compared to traditional teaching methods. A sample of 228 primary school students from Grades 5 and 6 was randomly assigned to one of two groups: a high-visual engagement group (VR with 360° video) or a low-visual engagement group (static video and textbook). The findings revealed that students in the high-visual engagement group achieved higher levels of intrinsic motivation and demonstrated greater learning improvements than their counterparts in the low-visual engagement group. Furthermore, the study identified negative user experiences as a significant factor moderating the connection between intrinsic motivation and learning outcomes. These results highlight the value of integrating VR into conventional teaching practices, showcasing its potential to enhance student engagement and improve educational outcomes in history and cultural studies.

Recommending suitable exercises and providing the reasons for these recommendations is a highly valuable task, as it can significantly improve students' learning efficiency. Nevertheless, the extensive range of exercise resources and the diverse learning capacities of students present a notable difficulty in recommending exercises. Collaborative filtering approaches frequently have difficulties in recommending suitable exercises, whereas deep learning methods lack explanation, which restricts their practical use. To address these issue, this paper proposes KG4EER, an explainable exercise recommendation with a knowledge graph. KG4EER facilitates the matching of various students with suitable exercises and offers explanations for its recommendations. More precisely, a feature extraction module is introduced to represent students' learning features, and a knowledge graph is constructed to recommend exercises. This knowledge graph, which includes three primary entities - knowledge concepts, students, and exercises - and their interrelationships, serves to recommend suitable exercises. Extensive experiments conducted on three real-world datasets, coupled with expert interviews, establish the superiority of KG4EER over existing baseline methods and underscore its robust explainability.

Knowledge management involves finding, expanding, and using knowledge in an organisation to achieve goals. Its role is crucial in higher education to improve problem-solving, research, and teaching by acquiring, sharing, and applying knowledge. Higher education institutions can promote knowledge management through Communities of Practice, but doing so remains challenging due to cultural, organisational, and technological reasons. We present findings of the first step of co-design workshops with authentic higher education teaching teams that sought to understand (a) their practices as a community and any motivators and impediments to their community development; (b) how they perceived the tools they use for knowledge management; and (c) the kinds of tools they believed could help them better conduct knowledge management and develop as Communities of Practice. Our findings suggested four essential design requirements and informed our development of a new tool to support the knowledge management needs of higher education teaching teams.

Automatic Essay Scoring (AES) is a well-established educational pursuit that employs machine learning to evaluate student-authored essays. While much effort has been made in this area, current research primarily focuses on either (i) boosting the predictive accuracy of an AES model for a specific prompt (i.e., developing prompt-specific models), which often heavily relies on the use of the labeled data from the same target prompt; or (ii) assessing the applicability of AES models developed on non-target prompts to the intended target prompt (i.e., developing the AES models in a cross-prompt setting). Given the inherent bias in machine learning and its potential impact on marginalized groups, it is imperative to investigate whether such bias exists in current AES methods and, if identified, how it intervenes with an AES model's accuracy and generalizability. Thus, our study aimed to uncover the intricate relationship between an AES model's accuracy, fairness, and generalizability, contributing practical insights for developing effective AES models in real-world education. To this end, we meticulously selected nine prominent AES methods and evaluated their performance using seven distinct metrics on an open-sourced dataset, which contains over 25,000 essays and various demographic information about students such as gender, English language learner status, and economic status. Through extensive evaluations, we demonstrated that: (1) prompt-specific models tend to outperform their cross-prompt counterparts in terms of predictive accuracy; (2) prompt-specific models frequently exhibit a greater bias towards students of different economic statuses compared to cross-prompt models; (3) in the pursuit of generalizability, traditional machine learning models (e.g., SVM) coupled with carefully engineered features hold greater potential for achieving both high accuracy and fairness than complex neural network models.

The recent large language models (LLMs), e.g., ChatGPT, have been able to generate human-like and fluent responses when provided with specific instructions. While admitting the convenience brought by technological advancement, educators also have concerns that students might leverage LLMs to complete their writing assignments and pass them off as their original work. Although many AI content detection studies have been conducted as a result of such concerns, most of these prior studies modeled AI content detection as a classification problem, assuming that a text is either entirely human-written or entirely AI-generated. In this study, we investigated AI content detection in a rarely explored yet realistic setting where the text to be detected is collaboratively written by human and generative LLMs (termed as hybrid text for simplicity). We first formalized the detection task as identifying the transition points between human-written content and AI-generated content from a given hybrid text (boundary detection). We constructed a hybrid essay dataset by partially and randomly removing sentences from the original student-written essays and then instructing ChatGPT to fill in for the incomplete essays. Then we proposed a two-step detection approach where we (1) separated AI-generated content from human-written content during the encoder training process; and (2) calculated the distances between every two adjacent prototypes (a prototype is the mean of a set of consecutive sentences from the hybrid text in the embedding space) and assumed that the boundaries exist between the two adjacent prototypes that have the furthest distance from each other. Through extensive experiments, we observed the following main findings: (1) the proposed approach consistently outperformed the baseline methods across different experiment settings; (2) the encoder training process (i.e., step 1 of the above two-step approach) can significantly boost the performance of the proposed approach; (3) when detecting boundaries for single-boundary hybrid essays, the proposed approach could be enhanced by adopting a relatively large prototype size (i.e., the number of sentences needed to calculate a prototype), leading to a 22% improvement (against the best baseline method) in the In-Domain evaluation and an 18% improvement in the Out-of-Domain evaluation.

Learning analytics (LA) holds a promise to transform education by utilizing data for evidence-based decision-making. Yet, its application in early childhood education (ECE) remains relatively under-explored. ECE plays a crucial role in fostering fundamental numeracy and literacy skills. While standardized tests was intended to be used to monitor student progress, they have been increasingly assumed summative and high-stake due to the substantial impact. The pressures in succeeding in such standardized tests have been well-documented to negatively affect both students and teachers. Attempting to ease such stress and better support students and teachers, the current study delved into the LA potential for predicting standardized test performance using formative assessments. Beyond predictive accuracy, the study addressed ethical considerations related to fairness to uncover potential risks associated with LA adoption. Our findings revealed a promising opportunity to empower teachers and schools with more time and room to help students better prepared based on predictions obtained earlier before standardized tests. Notably, bias can be significantly observed in predictions for students with disabilities even they have same actual competence compared to students without disabilities. In addition, we noticed that inclusion of demographic attribute had no significant impact on the predictive accuracy, and not necessarily exacerbate the overall predictive bias, but may significantly affect the predictions received by certain demographic subgroups (e.g., students with different types of disability).

We propose a learning analytics-based methodology for assessing the collaborative writing of humans and generative artificial intelligence. Framed by the evidence-centered design, we used elements of knowledge-telling, knowledge transformation, and cognitive presence to identify assessment claims; we used data collected from the CoAuthor writing tool as potential evidence for these claims; and we used epistemic network analysis to make inferences from the data about the claims. Our findings revealed significant differences in the writing processes of different groups of CoAuthor users, suggesting that our method is a plausible approach to assessing human-AI collaborative writing.

We report on the design and evaluation of a novel technique for analysing the sociocognitive nature of collaborative problem-solving—neural epistemic network analysis (NENA). NENA combines the computational power and representational ability of graph neural networks (GNNs) to naturally incorporate social and cognitive features in the analysis with the interpretative advantages of epistemic network analysis (ENA). Comparing NENA and ENA on two datasets from collaborative problem-solving contexts, we found that NENA improves upon ENA’s ability to distinguish between known subgroups in CPS data, while also improving the interpretability and explainability of GNN results.

To meet the growing demand for future professionals who can present information to an audience and create quality written products, educators are increasingly assigning writing assignments that require students to gather information from multiple sources, reorganise and reinterpret knowledge from source materials, and plan for rhetorical structure goals in order to meet the task requirements. When evaluating an essay coherence, scorers manually look for the presence of required rhetorical categories, which takes time. Supervised Machine Learning (ML) techniques have proven to be an effective tool for automatic detection of rhetorical categories that approximate students’ cognitive engagement with source information. Previous studies that addressed this problem used relatively small datasets and reported relatively low kappa scores for accuracy, limiting the use of such models in real-world scenarios. Moreover, to empower educators to effectively evaluate the overall quality of students’ writing, the associations between the sequential patterns of rhetorical categories in students’ writing and writing performance must be examined, which remains largely unexplored in educational domain. Therefore, to fill these gaps, our study aimed to i) investigate the impact of data augmentation approaches on the performance of deep learning algorithms in classifying rhetorical categories in student essays according to Bloom‘s taxonomy ii) and explore the sequential patterns of rhetorical categories in students’ writing that can influence writing performance. Our findings showed that deep learning-based model BERT on Easy Data Augmentation (EDA) based augmented data achieved 20% higher Cohen’s kappa than normal (non-augmented) data, and we discovered that students in different performance groups were statistically different in terms of rhetorical patterns. Our proposed study is valuable in terms of building a data analytic foundation that can be used to create formative feedback on students’ writings based on the patterns of rhetorical categories to improve essay quality.

Even though the engagement in self-regulated learning (SRL) has been shown to boost academic performance, SRL skills of many learners remain underdeveloped. They often struggle to productively navigate multiple cognitive, affective, metacognitive and motivational (CAMM) processes in SRL. To provide learners with the required SRL support, it is essential to understand how learners enact CAMM processes as they study. More research is needed to advance the measurement of affective and motivational processes within SRL, and investigate how these processes influence learners’ cognition and metacognition. With this in mind, we conducted a lab study involving 22 university students who worked on a 45-minute reading and writing task in digital learning environment. We used a wearable electroencephalogram device to record learner academic emotional and motivational states, and digital trace data to record learner cognitive and metacognitive processes. We harnessed time series prediction and explainable artificial intelligence methods to examine how learner’s emotional and motivational states influence their choice of cognitive and metacognitive processes. Our results indicate that emotional and motivational states can predict learners’ use of low cognitive, high cognitive and metacognitive processes with considerable classification accuracy (F1 > 0.73), and that higher values of interest, engagement and excitement promote cognitive processing.

Multi-Modal Large Language Models (MM-LLMs) have demonstrated powerful reasoning abilities in various visual question-answering tasks. However, they face the challenge of lacking rigorous reasoning and precise arithmetic, when solving geometry questions. To address this challenge, we propose a novel prompting method, namely Reason-and-Execute (R&E), to enhance the accuracy of solving geometry questions by MM-LLMs. Specifically, the R&E method includes two templates: reasoning template and execution template. We first adopt a reverse-thinking approach to construct a rigorous reasoning template so that it guides MM-LLMs to start reasoning from the most relevant domain knowledge of the question and ultimately identify the arithmetic requirements. We then make use of program-assisted thought to construct execution template in order to guide MM-LLMs to understand the arithmetic requirements from reasoning template and generate executable code block. The answer is finally obtained by executing the code block. We evaluate our prompting method on 9 models in answering questions on 6 datasets (including four geometry datasets and two science datasets) compared to Chain-of-Thought (CoT) and Program-Aided Language (PAL) prompting methods. R&E method shows up to 12.8% improvement compared to CoT and PAL, proving strong reasoning and arithmetic abilities for solving geometry questions of our method. Moreover, we further analyze the answering accuracy from the different perspectives on solving geometric questions, including domain knowledge, geometry shapes, question length, and language. Through multiple analysis, our method is able to enhance the ability of MM-LLMs to solve geometry questions.

Automatic Text Scoring (ATS) is a widely-investigated task in education. Existing approaches often stressed the structure design of an ATS model and neglected the training process of the model. Considering the difficult nature of this task, we argued that the performance of an ATS model could be potentially boosted by carefully selecting data of varying complexities in the training process. Therefore, we aimed to investigate the effectiveness of curriculum learning (CL) in scoring educational text. Specifically, we designed two types of difficulty measurers: (i) pre-defined, calculated by measuring a sample's readability, length, the number of grammatical errors or unique words it contains; and (ii) automatic, calculated based on whether a model in a training epoch can accurately score the samples. These measurers were tested in both the easy-to-hard to hard-to-easy training paradigms. Through extensive evaluations on two widely-used datasets (one for short answer scoring and the other for long essay scoring), we demonstrated that (a) CL indeed could boost the performance of state-of-the-art ATS models, and the maximum improvement could be up to 4.5%, but most improvements were achieved when assessing short and easy answers; (b) the pre-defined measurer calculated based on the number of grammatical errors contained in a text sample tended to outperform the other difficulty measurers across different training paradigms.

Effective exercise recommendation is crucial for guiding students' learning trajectories and fostering their interest in the subject matter. However, the vast exercise resource and the varying learning abilities of individual students pose a significant challenge in selecting appropriate exercise questions. Collaborative filtering-based methods often struggle with recommending suitable exercises, while deep learning-based methods lack explanation, limiting their practical adoption. To address these limitations, this paper proposes KG4Ex, a knowledge graph-based exercise recommendation method. KG4Ex facilitates the matching of diverse students with suitable exercises while providing recommendation reasons. Specifically, we introduce a feature extraction module to represent students' learning states and construct a knowledge graph for exercise recommendation. This knowledge graph comprises three key entities (knowledge concepts, students, and exercises) and their interrelationships, and can be used to recommend suitable exercises. Extensive experiments on three real-world datasets and expert interviews demonstrate the superiority of KG4Ex over existing baseline methods and highlight its strong explainability.

Educational feedback has been widely acknowledged as an effective approach to improving student learning. However, scaling effective practices can be laborious and costly, which motivated researchers to work on automated feedback systems (AFS). Inspired by the recent advancements in the pre-trained language models (e.g., ChatGPT), we posit that such models might advance the existing knowledge of textual feedback generation in AFS because of their capability to offer natural-sounding and detailed responses. Therefore, we aimed to investigate the feasibility of using ChatGPT to provide students with feedback to help them learn better. Our results show that i) ChatGPT is capable of generating more detailed feedback that fluently and coherently summarizes students' performance than human instructors; ii) ChatGPT achieved high agreement with the instructor when assessing the topic of students' assignments; and iii) ChatGPT could provide feedback on the process of students completing the task, which might benefit students developing learning skills.

Machine Learning (ML) techniques have been increasingly adopted to support various activities in education, including being applied in important contexts such as college admission and scholarship allocation. In addition to being accurate, the application of these techniques has to be fair, i.e., displaying no discrimination towards any group of stakeholders in education (mainly students and instructors) based on their protective attributes (e.g., gender and age). The past few years have witnessed an explosion of attention given to the predictive bias of ML techniques in education. Though certain endeavors have been made to detect and alleviate predictive bias in learning analytics, it is still hard for newcomers to penetrate. To address this, we systematically reviewed existing studies on predictive bias in education, and a total of 49 peer-reviewed empirical papers published after 2010 were included in this study. In particular, these papers were reviewed and summarized from the following three perspectives: (i) protective attributes, (ii) fairness measures and their applications in various educational tasks, and (iii) strategies for enhancing predictive fairness. These findings were summarized into recommendations to guide future endeavors in this strand of research, e.g., collecting and sharing more quality data containing protective attributes, developing fairness-enhancing approaches which do not require the explicit use of protective attributes, validating the effectiveness of fairness-enhancing on students and instructors in real-world settings.

Essay writing has become one of the most common learning tasks assigned to students enrolled in various courses at different educational levels, owing to the growing demand for future professionals to effectively communicate information to an audience and develop a written product (i.e. essay). Evaluating a written product requires scorers who manually examine the existence of rhetorical categories, which is a time-consuming task. Machine Learning (ML) approaches have the potential to alleviate this challenge. As a result, several attempts have been made in the literature to automate the identification of rhetorical categories using Rhetorical Structure Theory (RST). However, RST do not provide information regarding students’ cognitive level, which motivates the use of Bloom’s Taxonomy. Therefore, in this research we propose to: i) investigate the extent to which classification of rhetorical categories can be automated based on Bloom’s taxonomy by comparing the traditional ML classifiers with the pre-trained language model BERT, ii) explore the associations between rhetorical categories and writing performance. Our results showed that BERT model outperformed the traditional ML-based classifiers with 18% better accuracy, indicating it can be used in future analytics tool. Moreover, we found a statistical difference between the associations of rhetorical categories in low-achiever, medium-achiever and high-achiever groups which implies that rhetorical categories can be predictive of writing performance.

Feedback is an effective way to assist students in achieving learning goals. The conceptualisation of feedback is gradually moving from feedback as information to feedback as a learner-centred process. To demonstrate feedback effectiveness, feedback as a learner-centred process should be designed to provide quality feedback content and promote student learning outcomes on the subsequent task. However, it remains unclear how instructors adopt the learner-centred feedback framework for feedback provision in the teaching practice. Thus, our study made use of a comprehensive learner-centred feedback framework to analyse feedback content and identify the characteristics of feedback content among student groups with different performance changes. Specifically, we collected the instructors’ feedback on two consecutive assignments offered by an introductory to data science course at the postgraduate level. On the basis of the first assignment, we used the status of student grade changes (i.e., students whose performance increased and those whose performance did not increase on the second assignment) as the proxy of the student learning outcomes. Then, we engineered and extracted features from the feedback content on the first assignment using a learner-centred feedback framework and further examined the differences of these features between different groups of student learning outcomes. Lastly, we used the features to predict student learning outcomes by using widely-used machine learning models and provided the interpretation of predicted results by using the SHapley Additive exPlanations (SHAP) framework. We found that 1) most features from the feedback content presented significant differences between the groups of student learning outcomes, 2) the gradient boost tree model could effectively predict student learning outcomes, and 3) SHAP could transparently interpret the feature importance on predictions.

The primary aim of cognitive diagnosis is to predict students' performance and knowledge structures by analyzing their learning behavior and answering results, thereby enabling educators can provide personalized instruction. Scholars have proposed many cognitive diagnostic models. However, most of the models do not fully extract and utilize the relevant data and parameters of cognitive diagnosis. Moreover, some models only rely on artificially designed simple functions to analyze the cognitive process of students, which cannot fully capture the complex relationship between students and the exercises. To address these limitations, this paper proposes a neural cognitive diagnostic model named NCDFSA, which focuses on students' attention to knowledge concepts. The model utilizes neural networks to diagnose students' knowledge and considers students' implicit relationships with knowledge concepts. We introduce the concept of attention matrix(AM) and define the importance of knowledge concepts by the frequency of usage of knowledge concepts to improve the prediction effect. This paper compares NCDFSA with existing classical models on four real datasets and finds that the model has higher accuracy and rationality in predicting student performance.

The past few years have witnessed an explosion of attention given to the bias displayed by Machine Learning (ML) techniques towards different groups of people (e.g., female vs. male). Although ML techniques have been widely adopted in education, it remains largely unexplored that to what extent such ML bias manifests itself in this specific setting and how it can be reduced and eliminated. Given the increasing importance of ML techniques in empowering educators to teach effectively, this study aimed to quantify the characteristics of the original datasets that might be correlated with the subsequent predictive unfairness displayed by ML models. To this end, we empirically investigated two types of data biases (i.e., distribution bias and hardness bias) towards students of different sexes and first-language backgrounds across a total of five frequently-performed predictive tasks in education. Then, to improve ML fairness, we drew inspiration from the well-established research in Class Balancing Techniques (CBTs), where samples are generated/removed to alleviate the predictive disparity between different prediction classes. We proposed two simple but effective strategies to empower class balancing techniques for alleviating data biases and improving prediction fairness. Through extensive analyses and evaluations, we demonstrated that ML models may greatly improve prediction fairness (improvement up to 66%) with only a small sacrifice (less than 1%) in prediction accuracy by balancing the training data with the use of students’ demographic information and the overall hardness bias measure. All data and code used in this study are publicly accessible via https://github.com/lsha49/FairEdu.

Pretrained Language Models (PLMs), though popular, have been diagnosed to encode bias against protected groups in the representations they learn, which may harm the prediction fairness of downstream models. Given that such bias is believed to be related to the amount of demographic information carried in the learned representations, this study aimed to quantify the awareness that a PLM (i.e., BERT) has regarding people’s protected attributes and augment BERT to improve prediction fairness of downstream models by inhibiting this awareness. Specifically, we developed a method to dynamically sample data to continue the pretraining of BERT and enable it to generate representations carrying minimal demographic information, which can be directly used as input to downstream models for fairer predictions. By experimenting on the task of classifying educational forum posts and measuring fairness between students of different gender or first-language backgrounds, we showed that, compared to a baseline without any additional pretraining, our method improved not only fairness (with a maximum improvement of 52.33%) but also accuracy (with a maximum improvement of 2.53%). Our method can be generalized to any PLM and demographic attributes. All the codes used in this study can be accessed via https://github.com/lsha49/FairBERT_deploy.

Recommendation methods based on deep learning frameworks have drastically increased over recent years, covering virtually all the sub-topics in recommender systems. Among these topics, one-class collaborative filtering (OCCF) as a fundamental problem has been studied most extensively. However, most of existing deep learning-based OCCF methods are essentially focused on either defining new prediction rules by replacing conventional shallow and linear inner products with a variety of neural architectures, or learning more expressive user and item factors with neural networks, which may still suffer from the inferior recommendation performance due to the underlying preference assumptions typically defined on single items. In this paper, we propose to address the limitation and justify the capacity of deep learning-based recommendation methods by adapting the setwise preference to the underlying assumption during the model learning process. Specifically, we propose a new setwise preference assumption under the neural recommendation frameworks and devise a general solution named DeepSet, which aims to enhance the learning abilities of neural collaborative filtering methods by activating the setwise preference at different neural layers, namely 1) the feature input layer, 2) the feature output layer, and 3) the prediction layer. Extensive experiments on four commonly used datasets show that our solution can effectively boost the performance of existing deep learning based methods without introducing any new model parameters.

Existing research indicates that students prefer to work with tutors who express politely in online human-human tutoring, but excessive polite expressions might lower tutoring efficacy. However, there is a shortage of understanding about the use of politeness in online tutoring and the extent to which the politeness of instructional strategies can contribute to students’ achievement. To address these gaps, we conducted a study on a large-scale dataset (5,165 students and 116 qualified tutors in 18,203 online tutoring sessions) of both effective and ineffective human-human online tutorial dialogues. The study made use of a well-known dialogue act coding scheme to identify instructional strategies, relied on the linguistic politeness theory to analyse the politeness levels of the tutors’ instructional strategies, and utilised Gradient Tree Boosting to evaluate the predictive power of these politeness levels in revealing students’ problem-solving performance. The results demonstrated that human tutors used both polite and non-polite expressions in the instructional strategies. Tutors were inclined to express politely in the strategy of providing positive feedback but less politely while providing negative feedback and asking questions to evaluate students’ understanding. Compared to the students with prior progress, tutors provided more polite open questions to the students without prior progress but less polite corrective feedback. Importantly, we showed that, compared to previous research, the accuracy of predicting student problem-solving performance can be improved by incorporating politeness levels of instructional strategies with other documented predictors (e.g., the sentiment of the utterances).

Zijie Zeng, Xinyu Li, Dragan Gasevic, Guanliang Chen. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Learning Analytics and Knowledge (LAK) and Educational Data Mining (EDM) are two of the most popular venues for researchers and practitioners to report and disseminate discoveries in data-intensive research on technology-enhanced education. After the development of about a decade, it is time to scrutinize and compare these two venues. By doing this, we expected to inform relevant stakeholders of a better understanding of the past development of LAK and EDM and provide suggestions for their future development. Specifically, we conducted an extensive comparison analysis between LAK and EDM from four perspectives, including (i) the topics investigated; (ii) community development; (iii) community diversity; and (iv) research impact. Furthermore, we applied one of the most widely-used language modeling techniques (Word2Vec) to capture words used frequently by researchers to describe future works that can be pursued by building upon suggestions made in the published papers to shed light on potential directions for future research.

This paper presents an approach to automatic labeling of the content of messages in online discussion according to the categories of social presence. To achieve this goal, the proposed approach is based on a combination of traditional text mining features and word counts extracted with the use of established linguistic frameworks (i.e., LIWC and Coh-metrix). The best performing classifier obtained 0.95 and 0.88 for accuracy and Cohen's kappa, respectively. This paper also provides some theoretical insights into the nature of social presence by looking at the classification features that were most relevant for distinguishing between the different categories. Finally, this study adopted epistemic network analysis to investigate the structural construct validity of the automatic classification approach. Namely, the analysis showed that the epistemic networks produced based on messages manually and automatically coded produced nearly identical results. This finding thus produced evidence of the structural validity of the automatic approach.

Massive Open Online Courses (MOOCs) aim to educate the world, especially learners from developing countries. While MOOCs are certainly available to the masses, they are not yet fully accessible. Although all course content is just clicks away, deeply engaging with a MOOC requires a substantial time commitment, which frequently becomes a barrier to success. To mitigate the time required to learn from a MOOC, we here introduce a design that enables learners to earn money by applying what they learn in the course to real-world marketplace tasks. We present a Paid Task Recommender System (Rec-$ys), which automatically recommends course-relevant tasks to learners as drawn from online freelance platforms. Rec-$ys has been deployed into a data analysis MOOC and is currently under evaluation.

Social comparison theory asserts that we establish our social and personal worth by comparing ourselves to others. In in-person learning environments, social comparison offers students critical feedback on how to behave and be successful. By contrast, online learning environments afford fewer social cues to facilitate social comparison. Can increased availability of such cues promote effective self-regulatory behavior and achievement in Massive Open Online Courses (MOOCs)? We developed a personalized feedback system that facilitates social comparison with previously successful learners based on an interactive visualization of multiple behavioral indicators. Across four randomized controlled trials in MOOCs (overall N = 33, 726), we find: (1) the availability of social comparison cues significantly increases completion rates, (2) this type of feedback benefits highly educated learners, and (3) learners' cultural context plays a significant role in their course engagement and achievement.

MOOC research is typically limited to evaluations of learner behavior in the context of the learning environment. However, some research has begun to recognize that the impact of MOOCs may extend beyond the confines of the course platform or conclusion of the course time limit. This workshop aims to encourage our community of learning analytics researchers to examine the relationship between performance and engagement within the course and learner behavior and development beyond the course. This workshop intends to build awareness in the community regarding the importance of research measuring multi-platform activity and long-term success after taking a MOOC. We hope to build the community's understanding of what it takes to operationalize MOOC learner success in a novel context by employing data traces across the social web.

The rising number of Massive Open Online Courses (MOOCs) enable people to advance their knowledge and competencies in a wide range of fields. Learning though is only the first step, the transfer of the taught concepts into practice is equally important and often neglected in the investigation of MOOCs. In this paper, we consider the specific case of FP101x (a functional programming MOOC on edX) and the extent to which learners alter their programming behaviour after having taken the course. We are able to link about one third of all FP101x learners to GitHub, the most popular social coding platform to date and contribute a first exploratory analysis of learner behaviour beyond the MOOC platform. A detailed longitudinal analysis of GitHub log traces reveals that (i) more than 8% of engaged learners transfer, and that (ii) most existing transfer learning findings from the classroom setting are indeed applicable in the MOOC setting as well.

Massive Open Online Courses (MOOCs) have gained considerable momentum since their inception in 2011. They are, however, plagued by two issues that threaten their future: learner engagement and learner retention. MOOCs regularly attract tens of thousands of learners, though only a very small percentage complete them successfully. In the traditional classroom setting, it has been established that personality impacts different aspects of learning. It is an open question to what extent this finding translates to MOOCs: do learners' personalities impact their learning & learning behaviour in the MOOC setting? In this paper, we explore this question and analyse the personality profiles and learning traces of hundreds of learners that have taken a EX101x Data Analysis MOOC on the edX platform. We find learners' personality traits to only weakly correlate with learning as captured through the data traces learners leave on edX.