Knowledge tracing (KT) is a problem of modeling students’ knowledge states to predict their future performance by observing their historical learning interactions. The collection of educational data presents significant challenges, as students’ limited learning engagement restricts the generation of large-scale interaction data, while stringent privacy regulations further limit the availability of student learning sequences from online platforms. Hence, it is crucial to enhance the capabilities of deep learning-based KT (DLKT) models by constructing large-scale datasets through the integration of student interaction data across multiple subjects and sources. The success of ChatGPT demonstrates that the decoder-only Transformer architecture is highly effective in capturing complex information from large-scale sequential data. Against this background, we propose a novel decoder-only Transformer architecture-based model, named Unified DLKT ( UniKT ), to learn coherent and unified representations across a wide range of data sources. Specifically, we combine student learning sequences from six educational scenarios and utilize a multi-source encoding to learn unified representations of interactions from mixed data. UniKT is a stack of Transformer decoder layers for handling long-term dependencies among students’ historical interactions and future performance. We evaluate UniKT on six publicly available real-world educational datasets, and experimental results demonstrate that our method outperforms the majority of existing DLKT models in terms of AUC and accuracy. Furthermore, the empirical analysis shows the strong transferability and adaptability of UniKT in learning from multiple sources. To encourage reproducible research, we make our data and code publicly available at https://pykt.org/ .

The development of online intelligent educational systems has revolutionized personalized learning, presenting an opportunity for the integration of knowledge tracing (KT). KT is an essential task that leverages students’ historical interactions to model their knowledge states, enabling accurate predictions of their future performance. The application of deep learning models on KT tasks, also known as deep learning based knowledge tracing (DLKT) models, has accelerated the process of KT tasks in recent years. Although DLKT models have achieved promising results, there is a huge challenge to avoid models generating wrong estimations that yield bad guidance in educational contexts. Hence, in this work, we propose a simulation framework to explore the feasibility of minimizing the evaluation cost while guaranteeing prediction performance. In the framework, we initially design a simple yet efficient DLKT model that learns the actual process of students’ knowledge acquisition. We then select the reliable predictions generated by the proposed model and assign the unreliable ones to teaching professionals based on confidence estimations. We present the results of a proof-of-concept experiment on three real-world publicly available datasets to demonstrate that our framework can obtain the balance of human cost and automatic evaluation accuracy, which can be flexibly deployed to real-world educational contexts in the future. To encourage reproducible research, we make our code publicly available at https://github.com/gwbnwnwh/human-in-the-loop.

Accurately judging students' ongoing performance is very crucial for real-world educational scenarios. In this work, we focus on the task of automatically predicting students' levels of mastery of math questions from teacher-student classroom dialogue data in the online learning environment. We propose a novel neural network armed with a multi-granularity attention mechanism to capture the personalized pedagogical instructions from the very noisy teacher-student dialogue transcriptions. We conduct experiments on a real-world educational dataset and the results demonstrate the superiority and availability of our model in terms of various evaluation metrics.

Knowledge tracing (KT) is the problem of predicting students' future performance based on their historical interaction sequences. With the advanced capability of capturing contextual long-term dependency, attention mechanism becomes one of the essential components in many deep learning based KT (DLKT) models. In spite of the impressive performance achieved by these attentional DLKT models, many of them are often vulnerable to run the risk of overfitting, especially on small-scale educational datasets. Therefore, in this paper, we propose sparseKT, a simple yet effective framework to improve the robustness and generalization of the attention based DLKT approaches. Specifically, we incorporate a k-selection module to only pick items with the highest attention scores. We propose two sparsification heuristics: (1) soft-thresholding sparse attention and (2) top-K sparse attention. We show that our sparseKT is able to help attentional KT models get rid of irrelevant student interactions and improve the predictive performance when compared to 11 state-of-the-art KT models on three publicly available real-world educational datasets. To encourage reproducible research, we make our data and code publicly available at https://github.com/pykt-team/pykt-toolkit1..

Knowledge tracing (KT) is the problem of predicting students’ future performance based on their historical interactions with intelligent tutoring systems. Recent studies have applied multiple types of deep neural networks to solve the KT problem. However, there are two important factors in real-world educational data that are not well represented. First, most existing works augment input representations with the co-occurrence matrix of questions and knowledge components1 (KCs) but fail to explicitly integrate such intrinsic relations into the final response prediction task. Second, the individualized historical performance of students has not been well captured. In this paper, we proposed AT-DKT to improve the prediction performance of the original deep knowledge tracing model with two auxiliary learning tasks, i.e., question tagging (QT) prediction task and individualized prior knowledge (IK) prediction task. Specifically, the QT task helps learn better question representations by predicting whether questions contain specific KCs. The IK task captures students’ global historical performance by progressively predicting student-level prior knowledge that is hidden in students’ historical learning interactions. We conduct comprehensive experiments on three real-world educational datasets and compare the proposed approach to both deep sequential KT models and non-sequential models. Experimental results show that AT-DKT outperforms all sequential models with more than 0.9% improvements of AUC for all datasets, and is almost the second best compared to non-sequential models. Furthermore, we conduct both ablation studies and quantitative analysis to show the effectiveness of auxiliary tasks and the superior prediction outcomes of AT-DKT. To encourage reproducible research, we make our data and code publicly available at https://github.com/pykt-team/pykt-toolkit 2.

Online dialogic instructions are a set of pedagogical instructions used in real-world online educational contexts to motivate students, help understand learning materials, and build effective study habits. In spite of the popularity and advantages of online learning, the education technology and educational data mining communities still suffer from the lack of large-scale, high-quality, and well-annotated teaching instruction datasets to study computational approaches to automatically detect online dialogic instructions and further improve the online teaching effectiveness. Therefore, in this paper, we present a dataset of online dialogic instruction detection, DialogID, which contains 30,431 effective dialogic instructions. These teaching instructions are well annotated into 8 categories. Furthermore, we utilize the prevalent pre-trained language models (PLMs) and propose a simple yet effective adversarial training learning paradigm to improve the quality and generalization of dialogic instruction detection. Extensive experiments demonstrate that our approach outperforms a wide range of baseline methods. The data and our code are available for research purposes from: https://github.com/ai4ed/DialogID.