Understanding how tutoring unfolds requires capturing not only what tutors and students say, but also the multi-turn pedagogical strategies that structure their conversation. In this study, we analyze 77 online math tutoring sessions (about 40 hours of tutoring) using a combination of LLM-assisted annotation and statistical modeling. Building on utterance-level annotations with a taxonomy of tutor moves, we identify three recurrent latent states using a Hidden Markov Model: Inquiry Elicitation (dominated by prompting and probing student reasoning), Direct Instruction (centered on explanation and scaffolding), and Affective Support (characterized by praise and socioemotional support). Sequential pattern mining revealed that these states exhibit distinct instructional motifs—for example, Inquiry Elicitation involves repeated prompting, while Direct Instruction features sustained explanatory scaffolding. These dynamics were correlated with different levels of student engagement. Sessions starting with Inquiry Elicitation and ending with Affective Support elicited more student talk, while sustaining Direct Instruction was associated with a higher likelihood that students met tutoring dosage recommendations by returning for additional sessions. Together, these results suggest that the impact of tutoring may lie not just in which strategies tutors employ, but in how those strategies are ordered and coordinated over time—patterns that reveal the deeper conversational architecture shaping student engagement. More broadly, this work demonstrates how integrating generative AI annotation with advanced statistical modeling can scale analyses of tutoring dialogue and identify conversational practices that may prompt sustained engagement.

The arrival of generative AI in schools has altered the education landscape in myriad ways and raises issues from the pedagogical to the ethical. In this context, the perceptions of teachers regarding whether, what, and how students should be taught about AI are important to consider. For this study, we conducted semi-structured interviews with several dozen teachers, representing a wide variety of teaching contexts. Our analysis of the interviews revealed that most teachers support student learning about AI, but that their reasoning varied substantially. Rationales for supporting education around AI included encouraging best practices for AI use, preparing students for their futures, equipping students to critically evaluate AI output, and promoting academic integrity, among others. Other teachers were less supportive of AI education, worried that encouraging its use would engender over-reliance on it. Teachers also expressed a variety of perspectives on the ways in which AI might either help or harm learning, augmenting concerns about what and how students are taught about these tools. These educator perspectives constitute an important contribution toward the wider conversation about AI in schools.

L@S ’25, Palermo, Italy

Computer Science (CS) equity initiatives in high schools are increasingly common, but their evaluation remains challenging due to a lack of specialized assessment tools. Existing tools for general education evaluation often fail to address the unique aspects of CS education. This study addresses this gap by leveraging the Experience lens of the CAPE framework (Capacity, Access, Participation, and Experience) to develop a survey tool in collaboration with educators in a Researcher-Practitioner Partnership. These survey questions are designed to capture nuanced data about students' CS experiences, which generic survey tools could overlook.

ABSTRACT The quality of education hinges on the proficiency and training of educators. Due to the importance of teacher training, the innovative platform Teacher Moments creates simulated classroom scenarios. In this scenario‐based learning, confusion is an important indicator to detect users who struggle with the simulations. Through Teacher Moments, we gathered 7975 audio recording responses from participants who self‐labelled their recordings according to whether they sounded confused. Our dataset stands out for its size, for not including actor‐generated audio, and for measuring confusion, a neglected emotion in artificial intelligence (AI). Our experiments tested unimodal approaches and feature‐level, model‐level and decision‐level fusion. Feature‐level fusion demonstrated superior performance to unimodal methods, achieving a balanced accuracy of 0.6607 on the test set. This outcome highlights the necessity for further investigation in the overlooked area of confusion detection, particularly employing realistic datasets like the one used in this study and exploring new methods. Beyond teacher training, the insights of this research also extend to other directions, such as other professionals making critical decisions, user interface design or adaptive learning systems.

When using simulations to design and implement novice teacher practice, a teacher educator may be concerned about if what is technically possible in terms of generating feedback to novice teachers' responses is educationally purposeful to support their learning. This paper details the design of infrastructure to incorporate user feedback within the Teacher Moments platform that is generated by an AI agent, and how we designed to sustain and scale the expertise of mathematics teacher educators when training a large language model. To best support the learning of novice mathematics teacher users to enact ambitious and equitable mathematics teaching, this paper explains the research design of training a large language model by collaborating with mathematics teacher educators to edit or endorse generated feedback across multiple training cycles. This paper also describes the UI design to explore potential of hosting such processes all within the Teacher Moments platform.

Incorporating recordings of teacher-student conversations into the training of LLMs has the potential to improve AI tools. Although AI developers are encouraged to put "humans in the loop" of their AI safety protocols, educators do not typically drive the data collection or design and development processes underpinning new technologies. To gather insight into privacy concerns, the adequacy of safety procedures, and potential benefits of recording and aggregating data at scale to inform more intelligent tutors, we interviewed a pilot sample of teachers and administrators using a scenario-based, semi-structured interview protocol. Our preliminary findings reveal three "paradoxes" for the field to resolve to promote safe, fair, and trustworthy AI. We conclude with recommendations for education stakeholders to reconcile these paradoxes and advance the science of learning.

Improving equity in K-12 Computer Science (CS) education benefits from the collaboration of classroom teachers, school counselors, and school leaders. This paper presents the outcomes of a pilot program that brought together cross-functional teams consisting of CS teachers, school counselors, and administrators. Over the course of a year, these teams attended monthly, equity-focused workshops, leveraging pre-existing materials from affordable, high-quality, research-based programs. The use of these resources demonstrated benefits of sequencing and synthesizing existing programs. Evidence from surveys and interviews shows that the workshops promoted learning and fostered collaboration between the cross-functional teams that would not have happened otherwise. Participants were motivated by the program, and they generated ideas that turned into actionable projects to promote CS education equity in their schools. While the initiative was well received, areas for improvement were identified, particularly, in school recruitment, workshop structure, and evaluation. This pilot initiative demonstrates that equity-centered programs comprised of cross-functional teams can help achieve systemic improvement of CS education equity.

Role-plays of interpersonal interactions are essential to learning across professions, but effective simulations are difficult to create in typical learning management systems. To empower educators and researchers to advance simulation-based pedagogy, we have developed the Digital Clinical Simulation Suite (DCSS, pronounced "decks"), an open-source platform for rehearsing for improvisational interactions. Participants are immersed in vignettes of professional practice through video, images, and text, and they are called upon to improvisationally make difficult decisions through recorded audio and text. Tailored data displays support participant reflection, instructional facilitation, and educational research. DCSS is based on six design principles: 1) Community Adaptation, 2) Masked Technical Complexity, 3) Authenticity of Task, 4) Improvisational Voice, 5) Data Access through "5Rs", and 6) Extensible AI Coaching. These six principles mean that any educator should be able to create a scenario that learners should engage in authentic professional challenges using ordinary computing devices, and learners and educators should have access to data for reflection, facilitation, and development of AI tools for real-time feedback. In this paper, we describe the architecture of DCSS and illustrate its use and efficacy in cases from online courses, colleges of education, and K-12 schools.

Implementing high-quality professional learning on diversity, equity, and inclusion (DEI) issues is a massive scaling challenge. Integrating dynamic support using natural language processing (NLP) into equity teaching simulations may allow for more responsive, personalized training in this field. In this study, we trained machine learning models on participants' text responses in an equity teaching simulation (494 users; 988 responses) to detect certain text features related to equity. We then integrated these models into the simulation to provide dynamic supports to users during the simulation. In a pilot study (N = 13), we found users largely thought the feedback was accurate and incorporated the feedback in subsequent simulation responses. Future work will explore replicating these results with larger and more representative samples.

Practice is essential for learning. However, for many interpersonal skills, there often are not enough opportunities and venues for novices to repeatedly practice. Role-playing simulations offer a promising framework to advance practice-based professional training for complex communication skills, in fields such as teaching. In this work, we introduce ELK (Eliciting Learner Knowledge), a role-playing simulation system that helps K-12 teachers develop effective questioning strategies to elicit learners' prior knowledge. We evaluate ELK with 75 pre-service teachers through a mixed-method study. We find that teachers demonstrate a modest increase in effective questioning strategies and develop sympathy towards students after using ELK for 3 rounds. We implement a supplementary activity in ELK in which users evaluate transcripts generated from past role-play sessions. We have tentative evidence that a combination of role-play and evaluating conversation moves may be more effective for learning. We contribute design implications of using role-play systems for communication strategy training.

Massive Open Online Courses (MOOCs) have opened new educational possibilities for learners around the world. Most of the research and spotlight has been concentrated on a handful of global, English-language providers, but there are a growing number of regional providers of MOOCS in languages other than English. In this work, we have partnered with thirteen MOOC providers from around the world. We apply a multi-platform approach generating a joint and comparable analysis with data from millions of learners. This allows us to examine learning analytics trends at a macro level across various MOOC providers, with a goal of understanding which MOOC trends are globally universal and which of them are context-dependent. The analysis reports preliminary results on the differences and similarities of trends based on the country of origin, level of education, gender and age of their learners across global and regional MOOC providers. This study exemplifies the potential of macro learning analytics in MOOCs to understand the ecosystem and inform the whole community, while calling for more large scale studies in learning analytics through partnerships among researchers and institutions.

Digital clinical simulations (DCSs) are a promising tool for professional learning on diversity, equity, and inclusion (DEI) issues across a variety of fields. Although digital clinical simulations can be integrated into large-scale learning environments, less is known about how to design these types of simulations so they can scale effectively. We describe the results of two studies of a digital clinical simulation tool called Jeremy's Journal. In Study 1, we implemented this simulation in an in-person workshop with a human facilitator. We found that participants described their learning experiences positively and reported changes in attitudes. In Study 2, we used the simulation within an online course but replaced the human facilitator with an asynchronous, text-based adaptation of the facilitation script. Although learners in Study 2 described the experience in the simulation positively, we did not observe changes in attitudes. We discuss the implications of these findings for the design of DCSs at scale

The late 2000s and 2010s saw the full arc of a dramatic hype cycle in learning at scale, where charismatic technologists made bold and ultimately unfounded predictions about how technologies would disrupt schooling systems. Looking toward the 2020s, a more productive approach to learning at scale is the tinkerer's stance, one that emphasizes incremental improvements on the long history of learning at scale. This article offers two organizational constructs for navigating and building on that history. Classifying learning-at-scale technologies into three genres-instructor-guided, algorithm-guided, and peer-guided approaches-helps identify how emerging technologies build on prior efforts and throws into relief that which is genuinely new. Four as-yet intractable dilemmas-the curse of the familiar, the edtech Matthew effect, the trap of routine assessment, and the toxic power of data and experiments-offer a set of grand challenges that learning-at-scale tinkerers will need to tackle in order to see more dramatic improvements in school systems.

The relationship between pricing and learning behavior is an increasingly important topic in MOOC (massive open online course) research. We report on two case studies where cohorts of learners were offered coupons for free-certificates to explore price reductions might influence user behavior in MOOC-based online learning settings. In Case Study #1, we compare participation and certification rates between courses with and without coupons for free-certificates. In the courses with a free-certificate track, participants signed up for the verified certificate track at higher rates and completion rates among verified students were higher than in the paid-certificate track courses. In Case Study #2, we compare the behaviors of learners within the same courses based on whether they received access to a free-certificate track. Access to free-certificates was associated with somewhat lower certification rates, but overall certification rates remained high particularly among those who viewed the courses. These findings suggests that some other incentives, other than simply the sunk-cost of paying for a verified certificate-track, may motivate learners to complete MOOC courses.

While global massive open online course (MOOC) providers such as edX, Coursera, and FutureLearn have garnered the bulk of attention from researchers and the popular press, MOOCs are also provisioned by a series of regional providers, who are often using the Open edX platform. We leverage the data infrastructure shared by the main edX instance and one regional Open edX provider, Edraak in Jordan, to compare the experience of learners from Arab countries on both platforms. Comparing learners from Arab countries on edX to those on Edraak, the Edraak population has a more even gender balance, more learners with lower education levels, greater participation from more developing countries, higher levels of persistence and completion, and a larger total population of learners. This "apples to apples" comparison of MOOC learners is facilitated by an approach to multiplatform MOOC analytics, which employs parallel research processes to create joint aggregate datasets without sharing identifiable data across institutions. Our findings suggest that greater research attention should be paid towards regional MOOC providers, and regional providers may have an important role to play in expanding access to higher education.

In computer science classrooms, the assumptions teachers have about students can significantly shape their interactions. Deeper understandings of the decisions impacting equity offers teacher educators and researchers new leverage in cultivating equitable teaching. Our work uses interactive online practice spaces to focus on specific teaching decisions that may be affected by teachers' assumptions about students. Teacher practice spaces are learning experiences, inspired by games and simulations, that allow teachers to rehearse and reflect on important decisions in teaching. Practice spaces are a potentially powerful approach for encoding equitable teaching strategies because they have the potential to reveal the different assumptions and interpretations which drive different teaching decisions We developed these practice spaces and embedded them within CS teacher preparation programs where they have been used by over 6,000 teachers. In this workshop, we'll use online practice spaces as a novel way to approach discussions about equity in computer science classrooms. We'll have participants try out different variations on these practice spaces, brainstorm ideas for improving existing practice space, and invite reflection about challenges they've observed in training CS teachers in equitable teaching. Participants will leave with links to practice spaces and related curriculum materials they can use in their own work.

Over two iterations of a Massive Open Online Course (MOOC) for school leaders, Launching Innovation in Schools, we developed and tested design elements to support the transfer of online learning into offline action. Effective professional learning is job-embedded: learners should employ new skills and knowledge at work. We aimed to get participants to both plan and actually launch new change efforts, and a subset of our most engaged participants were willing to do so during the course. Assessments, instructor calls to action, and exemplars supported student actions. We found that participants led change initiatives, held stakeholder meetings, collected new data about their contexts, and shared and used course materials collaboratively. Collecting data about participant learning and behavior outside the MOOC environment is essential for researchers and designers looking to create effective online environments for professional learning.

Equity is a core component of many computer science teacher preparation programs. One promising approach is addressing unconscious bias in teachers, which may impact teacher expectations and interactions with students. Since early intervention literature indicates that asking individuals to suppress biases is counterproductive, our work uses online interactive case studies as practice spaces to focus on teaching decisions that may be impacted by unconscious bias. Our initial findings indicate that when embedded within teacher preparation programs, practice spaces produce rich learning opportunities, and our analysis yields insights into how beliefs or biases may interfere with principles of equity like disrupting preparatory privilege.

Equity is a core component of many computer science teacher preparation programs. One promising approach is addressing unconscious bias in teachers related to the race, ethnicity or gender of students. These biases may impact teacher expectations and interactions with students in a variety of classroom scenarios. Early literature on interventions targeting unconscious bias suggests that asking individuals to suppress biases is counterproductive. Our work uses the affordances of interactive online practice spaces to instead focus on specific teaching decisions that may be impacted by unconscious bias. We developed practice spaces and embedded them within CS teacher preparation programs. Our early findings indicate that practice spaces produce rich learning opportunities and analysis yields insight into what biases or beliefs may be interfering with teachers enacting principles of equity like disrupting preparatory privilege. In this workshop, we'll use online practice spaces to examine how we approach different classroom situations related to equity, and practice how we respond. We'll try two different variations on these practice spaces, and create space for participants to try a variety of other iterations on their own. We'll close by inviting folks to share their own stories of important classroom moments that problematized how they approached equitable teaching, and prototype creating practice spaces from those experiences. Participants will leave with links to practice spaces, and related curriculum materials they can use in CS teacher preparation courses, in teacher-led PLC groups, online CS teacher groups, or with local CSTA chapters.

Among all of the learners in Massive Open Online Courses (MOOCs) who intend to complete a course, the majority fail to do so. This intention-action gap is found in many domains of human experience, and research in similar goal pursuit domains suggests that plan-making is a cheap and effective nudge to encourage follow-through. In a natural field experiment in three HarvardX courses, some students received open-ended planning prompts at the beginning of a course. These prompts increased course completion by 29%, and payment for certificates by 40%. This effect was largest for students enrolled in traditional schools. Furthermore, the contents of students' plans could predict which students were least likely to succeed - in particular, students whose plans focused on specific times were unlikely to complete the course. Our results suggest that planning prompts can help learners adopted productive frames of mind at the outset of a learning goal that encourage and forecast student success.

We present Discourse, a tool for coding and annotating MOOC discussion forum data. Despite the centrality of discussion forums to learning in online courses, few tools are available for analyzing these discussions in a context-aware way. Discourse scaffolds the process of coding forum data by enabling multiple coders to work with large amounts of forum data. Our demonstration will enable attendees to experience, explore, and critique key features of the app.

Student intention and motivation are among the strongest predictors of persistence and completion in Massive Open Online Courses (MOOCs), but these factors are typically measured through fixed-response items that constrain student expression. We use natural language processing techniques to evaluate whether text analysis of open responses questions about motivation and utility value can offer additional capacity to predict persistence and completion over and above information obtained from fixed-response items. Compared to simple benchmarks based on demographics, we find that a machine learning prediction model can learn from unstructured text to predict which students will complete an online course. We show that the model performs well out-of-sample, compared to a standard array of demographics. These results demonstrate the potential for natural language processing to contribute to predicting student success in MOOCs and other forms of open online learning.

In this study, we develop methods for computationally measuring the degree to which students engage in MOOC forums with other students holding different political beliefs. We examine a case study of a single MOOC about education policy, Saving Schools, where we obtain measures of student education policy preferences that correlate with political ideology. Contrary to assertions that online spaces often become echo chambers or ideological silos, we find that students in this case hold diverse political beliefs, participate equitably in forum discussions, directly engage (through replies and upvotes) with students holding opposing beliefs, and converge on a shared language rather than talking past one another. Research that focuses on the civic mission of MOOCs helps ensure that open online learning engages the same breadth of purposes that higher education aspires to serve.

To minimize barriers to entry, massive open online course (MOOC) providers collect minimal demographic information about users. In isolation, these data are insufficient to address important questions about socioeconomic status (SES) and MOOC enrollment and performance. We demonstrate the use of third-party datasets to enrich demographic portraits of MOOC students and answer fundamental questions about SES and MOOC enrollment. We derive demographic information from registrants' geographic location by matching self-reported mailing addresses with data available from Esri at the census block group level and the American Community Survey at the zip code level. We then use these data to compare neighborhood income and parental education for US registrants in HarvardX courses to the US population as a whole. Overall, HarvardX registrants tend to reside in more affluent neighborhoods. Registrants on average live in neighborhoods with median incomes approximately. 45 standard deviations higher than the US population. Higher levels of parental education are also associated with a higher likelihood of registration.

Massive open online course (MOOC) platforms increasingly allow easily implemented randomized experiments. The heterogeneity of MOOC students, however, leads to two methodological obstacles in analyzing interventions to increase engagement. (1) Many MOOC participation metrics have distributions with substantial positive skew from highly active users as well as zero-inflation from high attrition. (2) High attrition means that in some experimental designs, most users assigned to the treatment never receive it; analyses that do not consider attrition result in "intent-to-treat" (ITT) estimates that underestimate the true effects of interventions. We address these challenges in analyzing an intervention to improve forum participation in the 2014 JusticeX course offered on the edX MOOC platform. We compare the results of four ITT models (OLS, logistic, quantile, and zero-inflated negative binomial regressions) and three "treatment-on-treated" (TOT) models (Wald estimator, 2SLS with a second stage logistic model, and instrumental variables quantile regression). A combination of logistic, quantile, and zero-inflated negative binomial regressions provide the most comprehensive description of the ITT effects. TOT methods then adjust the ITT underestimates. Substantively, we demonstrate that self-assessment questions about forum participation encourage more students to engage in forums and increases the participation of already active students.

We report on an experiment testing the effects of releasing all of the content in a Massive Open Online Course (MOOC) at launch versus in a staggered release. In 2013, HarvardX offered two "runs" of the HeroesX course: In the first, content was released weekly over four months; in the second, all content was released at once. We develop three operationalizations of "ontrackness" to measure how students participated in sync with the recommended syllabus. Ontrackness in both versions was low, though in the second, mean ontrackness was approximately one-half of levels in the first HeroesX. We find few differences in persistence, participation, and completion between the two runs. Controlling for a students' number of active weeks, we estimate modest positive effects of ontrackness on certification. The revealed preferences of students for flexibility and the minimal benefits of ontrackness suggest that releasing content all at once may be a viable strategy for MOOC designers.

In contrast to typical laboratory experiments, the everyday use of online educational resources by large populations and the prevalence of software infrastructure for A/B testing leads us to consider how platforms can embed in vivo experiments that do not merely support research, but ensure practical improvements to their educational components. Examples are presented of randomized experimental comparisons conducted by subsets of the authors in three widely used online educational platforms -- Khan Academy, edX, and ASSISTments. We suggest design principles for platform technology to support randomized experiments that lead to practical improvements -- enabling Iterative Improvement and Collaborative Work -- and explain the benefit of their implementation by WPI co-authors in the ASSISTments platform.

Massive Open Online Courses (MOOCs) employ a variety of components to engage students in learning (eg. videos, forums, quizzes). Some components are graded, which means that they play a key role in a student's final grade and certificate attainment. It is not yet clear how the due date structure of graded components affects student outcomes including academic performance and alternative modes of learning of students. Using data from HarvardX and MITx, Harvard's and MIT's divisions for online learning, we study the structure of due dates on graded components for 10 completed MOOCs. We find that stricter due dates are associated with higher certificate attainment rates but fewer students who join late being able to earn a certificate. Our findings motivate further studies of how the use of graded components and deadlines affects academic and alternative learning of MOOC students, and can help inform the design of online courses.