Many areas of educational research require the analysis of data that have an inherent sequential or temporal ordering. In certain cases, researchers are specifically interested in the transitions between different states—or events—in these sequences, with the goal being to understand the significance of these transitions; one notable example is the study of affect dynamics, which aims to identify important transitions between affective states. Unfortunately, a recent study has revealed a statistical bias with several metrics used to measure and compare these transitions, possibly causing these metrics to return unexpected and inflated values. This issue then causes extra difficulties when interpreting the results of these transition metrics. Building on this previous work, in this study we look in more detail at the specific mechanisms that are responsible for the bias with these metrics. After giving a theoretical explanation for the issue, we present an alternative procedure that attempts to address the problem with the use of marginal models. We then analyze the effectiveness of this procedure, both by running simulations and by applying it to actual student data. The results indicate that the marginal model procedure seemingly compensates for the bias observed in other transition metrics, thus resulting in more accurate estimates of the significance of transitions between states.

Retrieval practice (also known as testing effect or test-enhanced learning) is a well-studied and established technique for improving the retention of knowledge. Many previous works have confirmed the benefits of retrieval practice in laboratory experiments involving the memorization of words or facts. In this study, we build on these works and analyze retrieval practice in an intelligent tutoring system. Using a large data set composed of the actions of almost 4 million students studying math and chemistry, we look at the possible benefits of retrieval practice in the ALEKS adaptive learning and assessment system. We compare two different types of retrieval practice---one involving the assessment of learned material, and another involving the learning of closely related content that builds on the learned material---leveraging the scale of the available data to control for several confounding variables. Finally, we look at the timing of retrieval practice within the system and the possible effect it has on forgetting. The results indicate that a delay in retrieval practice is associated with better retention and that, while being assessed on learned material is beneficial, the learning of closely related content is associated with an even higher rate of retention.