Human tutoring interventions play a crucial role in supporting student learning, improving academic performance, and promoting personal growth. This paper focuses on analyzing mathematics tutoring discourse using talk moves—a framework of dialogue acts grounded in Accountable Talk theory. However, scaling the collection, annotation, and analysis of extensive tutoring dialogues to develop machine learning models is a challenging and resource-intensive task. To address this, we present SAGA22, a compact dataset, and explore various modeling strategies, including dialogue context, speaker information, pretraining datasets, and further fine-tuning. By leveraging existing datasets and models designed for classroom teaching, our results demonstrate that supplementary pretraining on classroom data enhances model performance in tutoring settings, particularly when incorporating longer context and speaker information. Additionally, we conduct extensive ablation studies to underscore the challenges in talk move modeling.

High-dosage tutoring has become an effective strategy for bolstering K-12 academic performance and combating education declines accelerated by the COVID-19 pandemic. To achieve high-dosage tutoring at scale, tutoring programs often rely on paraprofessional tutors—recruited tutors with college degrees who lack formal training in education—however, these tutors may require consistent and targeted feedback from instructional coaches for improvement. Accordingly, we developed a human-tutor coaching technology (HTCT) system to automatically extract discourse analytics pertaining to accountable talk moves (or academically productive talk) from tutoring sessions and provide feedback visualizations to coaches to aid their coaching sessions with tutors. We deployed HTCT in a user study using a virtual tutoring platform with 11 real coaches, 40 tutors, and their students to investigate coaches’ usage patterns with HTCT, perceptions of its utility, and changes in tutors’ talk. Overall, we found that coaches had positive perceptions of the system. We also observed an increase in accountable talk from tutors whose coaches used HTCT compared to tutors whose coaches did not. We discuss implications for AI-based applications which offer coaches a promising way to provide personalized, automated, and data-driven feedback to scale high-dosage tutoring.

This work investigates relationships between consistent attendance —attendance rates in a group that maintains the same tutor and students across the school year— and learning in small group tutoring sessions. We analyzed data from two large urban districts consisting of 206 9th-grade student groups (3 − 6 students per group) for a total of 803 students and 75 tutors. The students attended small group tutorials approximately every other day during the school year and completed a pre and post-assessment of math skills at the start and end of the year, respectively. First, we found that the attendance rates of the group predicted individual assessment scores better than the individual attendance rates of students comprising that group. Second, we found that groups with high consistent attendance had more frequent and diverse tutor and student talk centering around rich mathematical discussions. Whereas we emphasize that changing tutors or groups might be necessary, our findings suggest that consistently attending tutorial sessions as a group with the same tutor might lead the group to implicitly learn as a team despite not being one.

Transcripts of teaching episodes can be effective tools to understand discourse patterns in classroom instruction. According to most educational experts, sustained classroom discourse is a critical component of equitable, engaging, and rich learning environments for students. This paper describes the TalkMoves dataset, composed of 567 human-annotated K-12 mathematics lesson transcripts (including entire lessons or portions of lessons) derived from video recordings. The set of transcripts primarily includes in-person lessons with whole-class discussions and/or small group work, as well as some online lessons. All of the transcripts are human-transcribed, segmented by the speaker (teacher or student), and annotated at the sentence level for ten discursive moves based on accountable talk theory. In addition, the transcripts include utterance-level information in the form of dialogue act labels based on the Switchboard Dialog Act Corpus. The dataset can be used by educators, policymakers, and researchers to understand the nature of teacher and student discourse in K-12 math classrooms. Portions of this dataset have been used to develop the TalkMoves application, which provides teachers with automated, immediate, and actionable feedback about their mathematics instruction.

Engaging in physical computing activities involving both hardware and software provides a hands-on introduction to computer science. The move to remote learning for primary and secondary schools during the 2020-2021 school year due to COVID-19 made implementing physical computing activities especially challenging. However, it is important that these activities are not simply eliminated from the curriculum. This paper explores how a unit centered around students investigating how programmable sensors that can support data-driven scientific inquiry was collaboratively adapted for remote instruction. A case study of one teacher’s experience implementing the unit with a group of middle school students (ages 11 to 14) in her STEM elective class examines how her students could still engage in computational thinking practices around data and programming. The discussion includes both the challenges and unexpected affordances of engaging in physical computing activities remotely that emerged from her implementation.

This paper describes the design and classroom implementation of a week-long unit that aims to integrate computational thinking (CT) into middle school science classes using programmable sensor technology. The goals of this sensor immersion unit are to help students understand why and how to use sensor and visualization technology as a powerful data-driven tool for scientific inquiry in ways that align with modern scientific practice. The sensor immersion unit is anchored in the investigation of classroom data where students engage with the sensor technology to ask questions about and design displays of the collected data. Students first generate questions about how data data displays work and then proceed through a set of programming exercises to help them understand how to collect and display data collected from their classrooms by building their own mini data displays. Throughout the unit students draw and update their hand drawn models representing their current understanding of how the data displays work. The sensor immersion unit was implemented by ten middle school science teachers during the 2019/2020 school year. Student drawn models of the classroom data displays from four of these teachers were analyzed to examine students' understandings in four areas: function of sensor components, process models of data flow, design of data displays, and control of the display. Students showed the best understanding when describing sensor components. Students exhibited greater confusion when describing the process of how data streams moved through displays and how programming controlled the data displays.

Our work builds on advances in deep learning for natural language processing to automatically analyze transcribed classroom discourse and reliably generate information about teachers’ uses of specific discursive strategies called ”talk moves.” Talk moves can be used by both teachers and learners to construct conversations in which students share their thinking, actively consider the ideas of others, and engage in sustained reasoning. Currently, providing teachers with detailed feedback about the talk moves in their lessons requires highly trained observers to hand code transcripts of classroom recordings and analyze talk moves and/or one-on-one expert coaching, a time-consuming and expensive process that is unlikely to scale. We created a bidirectional long short-term memory (bi-LSTM) network that can automate the annotation process. We have demonstrated the feasibility of this deep learning approach to reliably identify a set of teacher talk moves at the sentence level with an F1 measure of 65%.

This experience report describes two iterations of a curriculum development process in which middle school teachers worked with our research team to collaboratively design and enact instructional units where students used sensors to investigate scientific phenomena. In this report, we examine the affordances of using a sensor platform to support the integration of disciplinary learning and computational thinking (CT) aligned with Next Generation Science Standards and the CT in STEM Taxonomy developed by Weintrop and colleagues. In the first unit, students investigated the conditions for mold growth within their school using a custom sensor system. After analyzing implementation experiences and student interest data, our team engaged in another round of co-design to develop a second instructional unit. This unit uses a different sensor system (the micro:bit) which supports additional CT in STEM practices due to its block-based programming interface and its real time data display. For the second unit we selected a different phenomenon: understanding and designing maglev trains.

Currently, providing teachers with detailed feedback about their classroom discourse strategies requires highly trained observers to hand code transcripts of classroom recordings to identify talk moves and/or one-on-one expert coaching. Both approaches are time-consuming and expensive, require considerable human expertise, and do not scale to large numbers of teachers. We are currently developing an innovative application, the TalkBack application, a new type of teacher learning environment based on the automated analysis of classroom recordings. The TalkBack application will utilize a big data infrastructure for managing and analyzing classroom recordings, including an embedded automated talk move classifier. The application will provide teachers with a detailed record of the discourse strategies used in their lessons. A central premise of our research is that this type of personalized, automated feedback can dramatically enhance teacher learning and support improvements in their instruction.The project will exemplify how next-generation repositories of classroom recordings can be architected to support large-scale research by enabling automated analyses based on machine learning models.