As AI tutoring systems are tested for use in K-12 classrooms, understanding their effects on learning and how interaction modality shapes student engagement is essential. Across three studies, students could ask for help from a generative AI tutor chatbot which responded in a text, audio, or video format during classroom math lessons. In the text modality, students experienced a typical text-only chatbot experience. In audio and video modalities, text was still present but each message was read aloud by an AI-generated voice and, in the case of video, accompanied by a human-like AI-generated avatar. In Study 1, we collected qualitative feedback on each modality. In Study 2, students were randomly assigned to one modality as they worked through solving math problems as part of normal course work. In Study 3, students were assigned a default modality but had the agency to switch modality. Results indicated a clear preference for text. Although initial engagement was higher in the audio condition, students frequently switched to text when given a choice. Student feedback and behavior also showed an aversion to the video modality. When modality was assigned, performance was also lowest in the video condition. However, when students could choose their modality, these differences disappeared. Instead, students who exercised agency over the modality engaged in longer conversations with the tutor, and the increased engagement fully mediated the effect of agency on accuracy. These findings suggest that giving students agency over modality can support sustained interactions with an AI tutor leading to higher performance.

Student motivation is essential for learning, especially in digital environments where students must regulate their own engagement. Reward systems are commonly used to encourage engagement, but these features can go unused, either because students do not find the rewards engaging or because they are not well understood. In the present study, we tested whether increased visibility of a reward system within a mathematics learning platform for middle-school students would drive increased engagement and whether this was associated with improved performance within the platform. Using log data from 28,749 students across the United States, we analyzed how the purchase and application of non-instructional rewards related to lesson completion, passing the first attempt, and passing after initial failure. In a series of path models, greater visibility significantly increased both purchasing and application of rewards which in turn increased completion rates of lessons. The relationship between applying rewards and passing the lesson varied by reward type. Opting to use rewards contingent on success predicted higher pass rates whereas reward use contingent on completion predicted lower pass rates. Overall, the findings suggested that engagement with a reward system can promote persistence, and the types of rewards learners choose can act as signals for different states (e.g., struggling). However, all of this is contingent on learners' understanding and perceived value of the system.

We developed the A.I. Math Personalization Tool (AMPT) to enhance cultural relevance in math word problems by giving students agency over the content. AMPT leverages generative AI to directly engage students as co-authors of math word problems. Through scaffolded conversations, the AI allows students to provide the context for a problem. Then, the AI integrates that context with the pedagogical standards of a target learning domain. We measured the attitudes of students towards mathematics before and after interaction with AMPT. After a single 30-min session co-authoring math word problems with AMPT, students’ sense of belonging in mathematics significantly increased, while other attitudes remained unchanged. AMPT provided students with the opportunity to express themselves and see their interests reflected in the math domain. After experiencing this level of agency over math content, their sense of belonging in mathematics increased. The results of this study demonstrate the potential for generative AI to enhance student choice, motivation, and, ultimately, achievement in mathematics.

Artificial intelligence (AI) applications to support human tutoring have potential to significantly improve learning outcomes, but engagement issues persist, especially among students from low-income backgrounds. We introduce an AI-assisted tutoring model that combines human and AI tutoring and hypothesize this synergy will have positive impacts on learning processes. To investigate this hypothesis, we conduct a three-study quasi-experiment across three urban and low-income middle schools: 1) 125 students in a Pennsylvania school; 2) 385 students (50% Latinx) in a California school, and 3) 75 students (100% Black) in a Pennsylvania charter school, all implementing analogous tutoring models. We compare learning analytics of students engaged in human-AI tutoring compared to students using math software only. We find human-AI tutoring has positive effects, particularly in student’s proficiency and usage, with evidence suggesting lower achieving students may benefit more compared to higher achieving students. We illustrate the use of quasi-experimental methods adapted to the particulars of different schools and data-availability contexts so as to achieve the rapid data-driven iteration needed to guide an inspired creation into effective innovation. Future work focuses on improving the tutor dashboard and optimizing tutor-student ratios, while maintaining annual costs per student of approximately $700 annually.

There is an increasing opportunity for AI-supported teacher-student orchestration. Preliminary evidence in small studies suggests that AI that better supports such coordination could lead to substantial student learning gains but much is unknown about how such orchestration might work at scale. In this work we focus on an existing tool, widely available to teachers as part of a commonly used math platform, to gain insights into how teachers perceive this tool and how they use a feature which allows them to mark when and why they help particular students as those students work on the software. Our teacher survey reveals that many teachers do use the tool's suggestion to inform which students to support, and our quantitative analysis of log files shows that when marking a student as helped, teachers most often report providing encouragement. Providing encouragement is far more frequent than providing direct math instruction, though when students are highlighted as being likely to fail the current section, teachers more frequently mark that they provided direct math help. Our work helps showcase the existing use and potential new directions for AI-supported teacher-student orchestration at scale.