Gaming the system, a behavior in which learners exploit a system's properties to make progress while avoiding learning, has frequently been shown to be associated with lower learning. However, when we applied a previously validated gaming detector across conditions in experiments with an algebra tutor, the detected gaming was not associated with reduced learning, challenging its validity in our study context. Our exploratory data analysis suggested that varying contextual factors across and within conditions contributed to this lack of association. We present a new approach, latent variable-based gaming detection (LV-GD), that controls for contextual factors and more robustly estimates student-level latent gaming tendencies. In LV-GD, a student is estimated as having a high gaming tendency if the student is detected to game more than the expected level of the population given the context. LV-GD applies a statistical model on top of an existing action-level gaming detector developed based on a typical human labeling process, without additional labeling effort. Across three datasets, we find that LV-GD consistently outperformed the original detector in validity measured by association between gaming and learning as well as reliability. LV-GD also afforded high practical utility: it more accurately revealed intervention effects on gaming, revealed a correlation between gaming and perceived competence in math and helped understand productive detected gaming behaviors. Our approach is not only useful for others wanting a cost-effective way to adapt a gaming detector to their context but is also generally applicable in creating robust behavioral measures.

Analytics of student learning data are increasingly important for continuous redesign and improvement of tutoring systems and courses. There is still a lack of general guidance on converting analytics into better system design, and on combining multiple methods to maximally improve a tutor. We present a multi-method approach to data-driven redesign of tutoring systems and its empirical evaluation. Our approach systematically combines existing and new learning analytics and instructional design methods. In particular, our methods involve identifying difficult skills and creating focused tasks for learning these difficult skills effectively following content redesign strategies derived from analytics. In our past work, we applied this approach to redesigning an algebraic modeling unit and found initial evidence of its effectiveness. In the current work, we extended this approach and applied it to redesigning two other tutor units in addition to a second iteration of redesigning the previously redesigned unit. We conducted a one-month classroom experiment with 129 high school students. Compared to the original tutor, the redesigned tutor led to significantly higher learning outcomes, with time mainly allocated to focused tasks rather than original full tasks. Moreover, it reduced over- and under-practice, yielded a more effective practice experience, and selected skills progressing from easier to harder to a greater degree. Our work provides empirical evidence of the effectiveness and generality of a multi-method approach to data-driven instructional redesign.

Educational content of today no longer only resides in textbooks and classrooms; more and more learning material is found in a free, accessible form on the Internet. Our long-standing vision is to transform this web of educational content into an adaptive, web-scale "textbook", that can guide its readers to most relevant "pages" according to their learning goal and current knowledge. In this paper, we address one core, long-standing problem towards this goal: identifying outcome and prerequisite concepts within a piece of educational content (e.g., a tutorial). Specifically, we propose a novel approach that leverages textbooks as a source of distant supervision, but learns a model that can generalize to arbitrary documents (such as those on the web). As such, our model can take advantage of any existing textbook, without requiring expert annotation. At the task of predicting outcome and prerequisite concepts, we demonstrate improvements over a number of baselines on six textbooks, especially in the regime of little to no ground-truth labels available. Finally, we demonstrate the utility of a model learned using our approach at the task of identifying prerequisite documents for adaptive content recommendation --- an important step towards our vision of the "web as a textbook".

Complex skill mastery requires not only acquiring individual basic component skills, but also practicing integrating such basic skills. However, traditional approaches to knowledge modeling, such as Bayesian knowledge tracing, only trace knowledge of each decomposed basic component skill. This risks early assertion of mastery or ineffective remediation failing to address skill integration. We introduce a novel integration-level approach to model learners' knowledge and provide fine-grained diagnosis: a Bayesian network based on a new kind of knowledge graph with progressive integration skills. We assess the value of such a model from multifaceted aspects: performance prediction, parameter plausibility, expected instructional effectiveness, and real-world recommendation helpfulness. Our experiments based on a Java programming tutor show that proposed model significantly improves two popular multiple-skill knowledge tracing models on all these four aspects.

Traditional Knowledge Tracing, which traces students' knowledge of each decomposed individual skill, has been a popular learner model for adaptive tutoring. Typically, a student is guided to the next skill when the student's knowledge on current skill is inferred as mastery. Unfortunately, this traditional approach no longer suffices to model complex skill practices where simple decompositions can not capture potential additional skills underlying the context as a whole. In such cases, mastery should only be granted when a student not only understands the basic of a skill but also can fluently apply a skill in varied application contexts. In this thesis, we aim to propose a data-driven approach to construct learner models considering different skill application contexts for tracing deeper knowledge, primarily based on Bayesian Networks. We aim to conduct novel, comprehensive, ``deep" evaluations, including internal data-drive evaluations, and external end-user evaluations examining the real world impact for students' personalized learning.

Various e-learning systems that provide electronic textbooks are gathering data on large numbers of student reading interactions. This data can potentially be used to model student knowledge acquisition. However, reading activity is often overlooked in canonical student modeling. Prior studies modeling learning from reading either estimate student knowledge at the end of all reading activities, or use quiz performance data with expert-crafted knowledge components (KCs). In this work, we demonstrate that the dynamic modeling of student knowledge is feasible and that automatic text analysis can be applied to save expert effort. We propose a data-driven approach for dynamic student modeling in textbook-based learning. We formulate the problem of modeling learning from reading as a reading-time prediction problem, reconstruct existing popular student models (such as Knowledge Tracing) and explore two automatic text analysis approaches (bag-of-words-based and latent semantic-based) to build the KC model. We evaluate the proposed framework using a dataset collected from a Human-Computer Interaction course. Results show that our approach for reading modeling is plausible; the proposed Knowledge Tracing-based student model reliably outperforms baselines and the latent semantic-based approach can be a promising way to construct a KC model. Serving as the first step to model dynamic knowledge in textbook-based learning, our framework can be applied to a broader context of open-corpus personalized learning.

Although the volume of online educational resources has dramatically increased in recent years, many of these resources are isolated and distributed in diverse websites and databases. This hinders the discovery and overall usage of online educational resources. By using linking between related subsections of online textbooks as a testbed, this paper explores multiple knowledge-based content linking algorithms for connecting online educational resources. We focus on examining semantic-based methods for identifying important knowledge components in textbooks and their usefulness in linking book subsections. To overcome the data sparsity in representing textbook content, we evaluated the utility of external corpuses, such as more textbooks or other online educational resources in the same domain. Our results show that semantic modeling can be integrated with a term-based approach for additional performance improvement, and that using extra textbooks significantly benefits semantic modeling. Similar results are obtained when we applied the same approach to other domains.