Madison Rose, Michael Bennie, Valeria Pagliai, Hatice Kubra Karakis, Qian Shen, Xinyi Tai, Walter L. Leite, Zoey Liu. Proceedings of the 30th Conference on Computational Natural Language Learning. 2026.

Advanced learning technologies (ALT) have become increasingly available to teachers for classroom use. Research has suggested that many factors can influence teacher adoption and fidelity of use of ALT in the classroom, including teacher beliefs, knowledge and experience, technological factors, and instructional factors. However, there has been scarce research linking teacher factors to student adoption of ALTs. This study examined the relationships between teacher characteristics and practices and student adoption and learning gains with a video recommendation system embedded within a virtual learning environment (VLE) for Algebra. Secondary data was obtained from an experimental study conducted over one academic semester in middle and high schools in a southeastern state of the United States. The sample included 52 teachers and 2936 students. The data included teacher responses to three surveys, and student demographic and achievement variables. A random forest was used to predict the rate that the students followed video recommendations in the VLE. The results show that the recommendation followed rate is related to the teachers' fidelity of use, frequency of student monitoring, and experience with the VLE. Most of the survey items specifically evaluating teachers' beliefs about the recommender were important predictors of students' following video recommendations. Teacher monitoring through a dashboard was the most important predictor. The analysis of treatment effect heterogeneity of the video recommendation system was performed using the generic machine learning inference (GenericML) method paired with random forests. Results show that teachers of students who benefitted most reported spending more time using the videos of the VLE and following student progress through the dashboard, but less time on the VLE than teachers of students who benefit the least. Teachers of students who benefitted the least had larger classrooms, struggled more with the challenges due to the Covid-19 pandemic, and spent less time with classroom planning. The results support the recommendation that teacher professional development for ALT should engage groups of educators in increasing their experience with the application so that they build comfort and confidence in its use in ways in which students are most likely to benefit.

This study presents a novel video recommendation system for an algebra virtual learning environment (VLE) that leverages ideas and methods from engagement measurement, item response theory, and reinforcement learning. Following Vygotsky's Zone of Proximal Development (ZPD) theory, but considering low affect and high affect students separately, we developed a system of five categories of video recommendations: 1) Watch new video; 2) Review current topic video with a new tutor; 3) Review segment of current video with current tutor; 4) Review segment of current video with a new tutor; 5) Watch next video in curriculum sequence. The category of recommendation was determined by student scores on a quiz and a sensor-free engagement detection model. New video recommendations (i.e., category 1) were selected based on a novel reinforcement learning algorithm that takes input from an item response theory model. The recommendation system was evaluated in a large field experiment, both before and after school closures due to the COVID-19 pandemic. The results show evidence of effectiveness of the video recommendation algorithm during the period of normal school operations, but the effect disappears after school closures. Implications for teacher orchestration of technology for normal classroom use and periods of school closure are discussed.

Discussion forums are important for students’ knowledge inquiry in online contexts, with help-seeking being an essential learning strategy in discussion forums. This study aimed to explore innovative methods to build a peer recommender that can provide fair and accurate intelligence to support help-seeking in online learning. Specifically, we have examined existing network embedding models, Node2Vec and FairWalk, to benchmark with the proposed fair network embedding (Fair-NE). A dataset of 187,450 post-reply pairs by 10,182 Algebra I students from 2015 to 2020 was sampled from Algebra Nation, an online algebra learning platform. The dataset was used to train and evaluate the engines of peer recommenders. We evaluated models with representation fairness, predictive accuracy, and predictive fairness. Our findings suggest that constructing fairness-aware models in learning analytics (LA) is crucial to tackling the potential bias in data and to creating trustworthy LA systems.

This study presents the results of a computational discourse analysis of discussion threads within an online Math tutoring platform. This work is theoretically motivated by prior work that established the importance of linguistic and semantic features in the discourse in mathematics education. The end goal of this study is to understand the characteristics of language that is produced and used within a discussion board for math. The discussion board corpus comprises of posts from 4,720 students, teachers, and study experts who interacted within an online teaching and learning tutoring platform for math. Linguistic profiles of the discussion board discourse were estimated using Principal Component Analysis (PCA) based on Coh-Metrix linguistic features related to cohesion, language sophistication, and lexical characteristics. The PCA analysis yielded seven Math Discourse Linguistic Components, which collectively explained 49% of the variance in the dataset. Theoretical and conceptual validation of components revealed that the linguistic features align with the communication goal and the nature of mathematics. The linguistic profiles that characterized the discussion board discourse included referential cohesion, information density, instructional language, lexical variation, compare and contrast devices, explicit relations devices, and syntactic complexity. The dominance of cohesive cues within the linguistic profiles demonstrate the communication goals within the Math discourse such as elaboration, providing instruction, compare and contrast, establishing explicit relations, and presenting information. As such, these components characterize the Math Discussion Board discourse in terms of variations in cohesive and task-oriented cues within communication among students.

Previous research has shown that providing video recommendations to students in virtual learning environments implemented at scale positively affects student achievement. However, it is also critical to evaluate whether the treatment effects are heterogeneous, and whether they depend on contextual variables such as disadvantaged student status and characteristics of the school settings. The current study extends the evaluation of a novel video recommendation system by performing an exploratory search for sources of heterogeneity of treatment effects. This study's design is a multi-site randomized controlled trial with an assignment at the student level across three large and diverse school districts in the southeast United States. The study occurred in Spring 2021, when some students were in regular classrooms and others in online classrooms. The results of the current study replicate positive effects found in a previous field experiment that occurred in Spring 2020, at the onset of the COVID-19 pandemic. Then, causal forests were used to investigate the heterogeneity of treatment effects. This study contributes to the literature on content sequencing systems and recommendation systems by showing how these systems can disproportionally benefit the groups of students who had higher levels of previous algebra ability, followed more recommendations, learned remotely, were Hispanic, and received free or reduced-price lunch, which has implications for the fairness of implementation of educational technology solutions.

To support online learners at a large scale, extensive studies have adopted machine learning (ML) techniques to analyze students’ artifacts and predict their learning outcomes automatically. However, limited attention has been paid to the fairness of prediction with ML in educational settings. This study intends to fill the gap by introducing a generic algorithm that can orchestrate with existing ML algorithms while yielding fairer results. Specifically, we have implemented logistic regression with the Seldonian algorithm and compared the fairness-aware model with fairness-unaware ML models. The results show that the Seldonian algorithm can achieve comparable predictive performance while producing notably higher fairness.