High-dosage tutoring has become an effective strategy for bolstering K-12 academic performance and combating education declines accelerated by the COVID-19 pandemic. To achieve high-dosage tutoring at scale, tutoring programs often rely on paraprofessional tutors—recruited tutors with college degrees who lack formal training in education—however, these tutors may require consistent and targeted feedback from instructional coaches for improvement. Accordingly, we developed a human-tutor coaching technology (HTCT) system to automatically extract discourse analytics pertaining to accountable talk moves (or academically productive talk) from tutoring sessions and provide feedback visualizations to coaches to aid their coaching sessions with tutors. We deployed HTCT in a user study using a virtual tutoring platform with 11 real coaches, 40 tutors, and their students to investigate coaches’ usage patterns with HTCT, perceptions of its utility, and changes in tutors’ talk. Overall, we found that coaches had positive perceptions of the system. We also observed an increase in accountable talk from tutors whose coaches used HTCT compared to tutors whose coaches did not. We discuss implications for AI-based applications which offer coaches a promising way to provide personalized, automated, and data-driven feedback to scale high-dosage tutoring.

Collaboration analytics has the potential to empower teachers and students with valuable insights to facilitate more meaningful and engaging collaborative learning experiences. Towards this end, we developed computational models of student speech during small group work, identifying instances of uplifting behavior related to three Community Agreements: community building, moving thinking forward, and being respectful. Pre-trained RoBERTa language models were fine-tuned and evaluated on human annotated data (N = 9,607 student utterances from 100 unique 5-minute classroom recordings). The models achieved moderate accuracies (AUROCs between 0.67-0.84) and were robust to speech recognition errors. Preliminary generalizability studies indicated that the models generalized well to two other domains (transfer ratios between 0.46-0.85; with 1.0 indicating perfect transfer). We also developed four approaches to provide qualitative feedback in the form of noticings (i.e., specific exemplars) of positive instances of the Community Agreements, finding moderate alignment with human ratings. This research contributes to the computational modeling of the relationship dimension of collaboration from noisy classroom data, selection of positive examples for qualitative feedback, and towards the empowerment of teachers to support diverse learners during collaborative learning.

Teacher support is pivotal for promoting collaborative learning. Students often face challenges in self-regulating their learning within small group settings, especially when presented with open-ended learning opportunities. Analyzing 542 moments of support in small group work with the MOSAIC protocol during a unit on physical computing suggests a relationship between support type and activity design. Students showed more initiative in seeking support in more complex tasks. Teacher support focused on small groups rather than the entire class during complex or open-ended activities. Support focus on the direction of the task is the most common type across all activities. Support for collaboration was most frequent in the activity requiring students to coordinate knowledge developed individually. This analysis provides insights into potential areas for enhancing student support in a CS classroom, particularly for collaboration.

With the proliferation of 3D printing technologies in schools and makerspaces, there is a need for teaching 3D modeling to students. Learning 3D modeling enhances spatial thinking skills, an essential skill for success in STEM. Creating 3D models requires students to have a deep understanding of 3D space, including rotating and scaling. In this study, we propose a framework developed through video-coding from analyzing screen recordings of middle-school students’ usage of a 3D modeling tool - Tinkercad. The proposed framework focuses on identifying challenges students encounter during 3D modeling. These challenges include spatial thinking skills, working with the Tinkercad interface, and mental model formation. We authenticated the framework by collecting and analyzing data from a 3D printing unit in three middle schools. Our results and subsequent analysis can guide educators and researchers on how to use this framework to support students in having productive learning experiences with Computer-Aided Design tools.

Engaging in physical computing activities involving both hardware and software provides a hands-on introduction to computer science. The move to remote learning for primary and secondary schools during the 2020-2021 school year due to COVID-19 made implementing physical computing activities especially challenging. However, it is important that these activities are not simply eliminated from the curriculum. This paper explores how a unit centered around students investigating how programmable sensors that can support data-driven scientific inquiry was collaboratively adapted for remote instruction. A case study of one teacher’s experience implementing the unit with a group of middle school students (ages 11 to 14) in her STEM elective class examines how her students could still engage in computational thinking practices around data and programming. The discussion includes both the challenges and unexpected affordances of engaging in physical computing activities remotely that emerged from her implementation.

Drag and drop programming languages have been promoted for broadening participation and increasing motivation for middle grade students in computer science (CS), yet their impact on motivation and the implications for equity are not well documented. This raises questions about potential issues of equity. Despite considerations to increase participation and diversity, these tools have not been evaluated as to how they provide more equitable outcomes regarding student motivation in CS. This study analyzes data from a survey (n=9,573) to investigate middle school student motivation in CS when considering previous experiences with drag and drop programming. Student responses to motivational items are categorized into four factors based on exploratory and confirmatory factor analysis. Multiple regression interaction analysis shows that previous experience is associated with increases (p<0.01) in three motivational factors: general academic and computer use confidence, programming confidence, and interest in future CS courses. These associations hold regardless of race and gender although they are lower for demographics underrepresented in CS, female and/or non-White and non-Asian students. Previous experience was also associated with an increase in interest in the current course for White and Asian male students and a decrease for female underrepresented minority students. While it appears that drag and drop programming efforts could help increase future interest and confidence for all students, it does so to differing degrees for different demographics. This suggests that these initiatives are not addressing issues of equity since groups overrepresented in CS still report higher gains than underrepresented demographics after early programming experiences.

Previous experience correlates to student retention in computer science at the secondary and undergraduate levels, it's impact in middle school is less well understood. Previous research findings from a University of Colorado large scale survey of students' conceptions of computer science have shown that there is a pervasive gender gap in middle school student confidence and interest in computer science. This study conducts a follow up investigation, analyzing data from fall 2014 to spring 2016 (n=6,128), using multiple regression analysis to investigate how student responses to motivational items concerning both confidence and interest vary by gender, previous experience with computer science, minority status, and having a computer at home (a proxy for privilege). Results show statistically significant associations (p<0.001) between the both outcomes (confidence and interest) and each of the four predictors. Gender had the largest association with each outcome. Students with previous experience with computer science also report higher confidence. This is equivalent in size to the higher confidence associated with having a computer at home and twice the higher confidence associated with being non-minority. For interest, previous experience associations were smaller but still significant. These findings imply that previous experience with computer science at the middle school level help to reduce the motivational differences by gender, privilege, and ethnicity.

This paper compares student outcomes from 75 K-12 teachers who participated in either online, blended, or face-to-face professional development design to support teacher implementation of a programming curriculum during the regular school day. The results are based on survey responses collected over two years from 4,832 students. With only one exception, the results showed no negative student outcomes when comparing student survey results from teachers who participated in online professional development compared to students of teachers who participated in face-to-face professional development. Students who had teachers who participated in face-to-face professional development, however, expressed stronger interest in designing their own games at home. These results suggest that online professional development that is designed to support K-12 teacher classroom implementation of CS education curricula is a viable model with respect to student outcomes. Recommendations for the design of online curricula for CS education are discussed.

As teacher professional development efforts continue to scale to meet the growing demand for computer science educators, more individuals with little formal preparation in the discipline are seeking training. For this reason, it is important to understand the kinds of questions that these teachers, along with administrators and IT staff, are asking professional development providers. This research examines questions received by Scalable Game Design (SGD), a project that engages students in computing through game and simulation design activities. SGD has provided training for more than 300 teachers since 2009. We have found several dimensions of interest in the question pool. The major categories of questions received are pedagogical, technical, and procedural, and we present a generalizable taxonomy of question types plus specific examples. In addition, question content and questioner roles fluctuate throughout the professional development and classroom implementation cycle, from initial inquiries to requests regarding next steps. Knowledge of these factors can lead to adjustments in training content to reduce questions later on and potentially increase teacher confidence; moreover, this information can inform planning for support resources, an especially important activity when project growth is forecast. We also discuss how our support structures have changed over the life of the project and compare them to two other curriculum initiatives that rely on unique software environments. Our future work will investigate the characteristics of teachers who are asking particular kinds of questions, including their disciplinary backgrounds, experience in the classroom, familiarity with SGD, and instructional environments.