Students' effort and emotions are important contributors to math learning. In a recent study evaluating the efficacy of MathSpring, a scalable web-based intelligent tutoring system that provides students with personalized math problems and affective support, system usage data were collected for 804 U.S. 10-12-year-olds. To understand the patterns in students' interactions with MathSpring and how patterns vary across students, hierarchical cluster heatmap analysis was performed. Guided by the patterns from the heatmap, latent profile analysis was conducted to identify student subgroups. Both analyses indicated that there were two groups of students: ''Confident Solvers'' who solved problems on their first attempt and reported high confidence, and ''Struggling Solvers'' who gave up, did not read problems, and reported higher frustration. These analyses provide insight into students' behaviors in online learning environments that can be scaled to students nationwide. Developers can use these insights to design systems that respond to students' usage and inform teachers about students' effort and emotions.

Self-regulated learning (SRL), or students actively managing their own learning, involves motivational beliefs, affect, effort, and detecting and evaluating mistakes. Students need feedback to engage in self-directed learning; scalable educational technology tools that offer formative assessment can allow students to evaluate their progress and adjust their strategies for effective math problem-solving without the need for teachers to provide feedback, fostering SRL at scale. This qualitative study examined whether a widely-used formative assessment math technology platform, ASSISTments, improved U.S. 11--14-year-old students' SRL. This study occurred in the context of a nationwide impact study examining the effects of ASSISTments with a professional learning community for teachers on students' math achievement. We used interview data from 33 teachers across two cohorts collected as part of the impact study to understand teachers' perspectives on their students' SRL development. Findings revealed that most teachers felt that using ASSISTments improved their students' accountability over their work, effort while solving math problems, motivation for math, and positive beliefs about the importance of making mistakes. This study leveraged teacher perspectives to provide insights into potential effects of interventions that may otherwise not be captured, highlighting a potential mechanism through which students' SRL can be improved nationwide.

AI technology-based educational programs have been shown to support students' learning. However, to address infrastructure, operational, sustainability, and training barriers when scaling interventions to schools, districts, institutions, and states, cost effectiveness should be considered as part of design, development, and implementation. We consider the cost of the ASSISTments intervention during a large-scale implementation as a part of an efficacy study. ASSISTments is an AI-enhanced educational tool that allows teachers to assign math work and provides students with supports for learning. The intervention was implemented in U.S. 7th grade (age 12--13 years) math classes with 4000+ students, and the long-term effects on math learning were measured one year later at the end of 8th grade (age 13--14 years). To estimate the implementation cost and contextualize the detected effects, we conducted a cost analysis by: (a) identifying the ''ingredients'' or components required, (b) determining the costs of these components, and (c) calculating the total program cost and the average cost per participant. The estimated cost is about 46.23 per student for an average long-term effect size of 0.10, placing ASSISTments at the lower end of cost relative to other interventions. Cost-effectiveness should be treated as an essential design and evaluation parameter from the earliest stages of development to ensure that AI-enabled technologies can reach students at scale in a fiscally responsible manner.

research-article Share on Student Effort and Progress Learning Analytics Data Inform Teachers' SEL Discussions in Math Class Authors: Natalie Brezack WestEd, USA WestEd, USA 0000-0001-5236-9721View Profile , Wynnie Chan WestEd, USA WestEd, USA 0009-0001-9851-192XView Profile , Mingyu Feng WestEd, USA WestEd, USA 0000-0001-9635-1611View Profile Authors Info & Claims LAK '24: Proceedings of the 14th Learning Analytics and Knowledge ConferenceMarch 2024Pages 338–348https://doi.org/10.1145/3636555.3636888Published:18 March 2024Publication History 0citation41DownloadsMetricsTotal Citations0Total Downloads41Last 12 Months41Last 6 weeks22 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Publisher SiteGet Access

Many districts in the U.S. are investing in education technologies to improve student learning. Yet, when technologies with established promise of evidence are deployed at scale, they frequently encounter challenges that compromise their efficacy. MathSpring is a technology-based math learning platform that offers personalized content, remedial tutoring, and affective support for students and reports for teachers. In a pilot study involving monthly usage with a researcher in the room, MathSpring showed promise of improving student learning. Following the pilot study, our team examined the efficacy of the intervention in a randomized controlled trial (RCT) with 64 fifth- and sixth-grade math teachers (34 treatment, 30 control) in 47 schools. Despite extensive training and support for teachers, usage was lower than expected. This paper presents an overview of the intervention and explores the challenges teachers faced to implement MathSpring. We discuss the factors that influence wide classroom adoption of technologies like MathSpring in a post-pandemic educational landscape.