Students struggle with accurately assessing their own performance, especially given little training to do so.We propose an AI-powered training tool to help students improve "metacognitive calibration, " or the ability to accurately predict their own learning, potentially enhancing learning outcomes by enabling students' use of metacognitioninformed learning behaviors.We present results from a randomized controlled trial (N = 133) assessing the effectiveness of the tool in a college-level computer-based learning environment.The AIdriven tool significantly improved learning gains compared to the control group by 8.9% (t = -2.384,p = .019),and this effect was significantly mediated by learning behaviors.Overconfident students who received the intervention showed significantly greater metacognitive calibration improvement than the control group by 4.1% (t = 2.001, p = .049).These insights highlight the value of AIpowered metacognitive calibration training and the importance of promoting specific metacognition-informed learning behaviors in computer-based learning.

Metacognition is a valuable tool for learning, since it is closely related to self-regulation and awareness of one's own affect. However, methods for automatically detecting and studying metacognition are scarce. Thus, in this paper we describe an algorithm for automatic detection of metacognitive language in writing. We analyzed text from the forums of two online, university-level science courses, which revealed common patterns of phrases that we used for automatic metacognition detection. The algorithm we developed exhibited high accuracy on expert-labeled metacognitive phrases (Spearman's rho = 0.878 and Cohen's kappa = 0.792), and provides a reliable, fast method for automatically annotating text corpora that are too large for manual annotation. We applied this algorithm to analyze relationships between students' metacognitive language and their academic performance, finding small correlations with course grade and medium-sized differences in metacognition across courses. We discuss how our algorithm can be used to advance metacognitive studies and online educational systems.