Problem-solving approaches are an essential part of learning. Knowing how students approach solving problems can help instructors improve their instructional designs and effectively guide the learning process of students. We propose a natural language processing (NLP) driven method to capture online learners’ problem-solving approaches at scale while using Massive Open Online Courses (MOOCs) as a learning platform. We employ an online survey to gather data, NLP techniques, and existing educational theories to investigate this in the lens of both computer science and education. The paper shows how NLP techniques, i.e. preprocessing, topic modeling, and text summarization, must be tuned to extract information from a large-scale text corpus. The proposed method discovered 18 problem-solving approaches from the text data, such as using pen and paper, peer learning, trial and error, etc. We also observed topics that appear over the years, such as clarifying code logic, watching videos, etc. We observed that students heavily rely on "tools" for solving programming problems and can expect that such selection of methods can vary depending on the type of task.

Learner data from multiple instances of two introductory Python MOOCs, offered before and during the COVID-19 pandemic, were analyzed to see if and how learner behavior and overall learning outcomes changed during the pandemic. We found a surge in interest during the pandemic, but the certification rate fell short of the pre-pandemic average. Moreover, this drop in certification rate is more noticeable for the advanced course in the sequence, and is more pronounced in groups traditionally underrepresented in computer-science and coding communities. Additionally, we analyzed learners' interaction with one another on the course discussion forum, and performed a sentiment analysis of the forum comments using natural language processing, which revealed behavioral trends that differed significantly during the pandemic.

Online education, and MOOCs in particular, experienced a dramatic rise during the COVID-19 lockdown. Many had extra time to start learning new topics, while a significant fraction of the population experienced disruptions in areas such as healthcare, childcare, and potential loss of livelihood, among others. In this work we analyze learner data from multiple instances of two introductory Python MOOCs, offered before and during the COVID-19 pandemic, to understand how the pandemic affected learner progress and outcomes in these courses. We explore multiple measures of COVID-19 severity, and find a strong correlation between a measure of severity and relative change in certification rate. Specifically, we find that the change in certification rate relative to pre-pandemic baselines showed a negative correlation as COVID-19 severity increased.

Student reflection is thought to be an important part of retaining and understanding knowledge gained in a course. Using natural language processing, we analyze and interpret student reflections from Massive Open Online Courses (MOOCs) to understand the students' sentiments and problem-solving procedures. The reflections are free text responses to questions from MIT 6.00.1x, an introductory programming MOOC. We compare different sentiment analysis methods, and conclude that the best-performing methods can robustly classify sentiment of student responses. In addition, we develop methods to analyze student problem-solving procedures using sentence parsing and topic modeling. We find our method can distinguish some common problem-solving procedures such as utilizing course resources.

Massive Open Online Courses (MOOCs) are accessible to anyone with a device that can connect to the internet. MOOCs aim to increase the accessibility of higher-level knowledge and skills, such as programming. To understand how students are performing and struggling in the course, we investigate a popular MITx MOOC that teaches introductory programming. We look at problem set questions and examine students with different levels of pre-existing knowledge. Specifically, we study the number of attempts of each group per question and the mean final accuracy of each group per question. We find that for nearly all questions, students with no programming experience struggle more than students with prior programming experience. Moreover, we observe a potential turning point in the course where students of all experience levels begin to struggle. Our findings both show that two groups of MOOC students perform differently and inform question design in MOOCs by demonstrating which question types are particularly arduous.

In classrooms, instructors teaching students how to code have the ability to monitor progress and provide feedback through regular interaction. There is generally no analogous tracing of learning progression in programming MOOCs, hindering the ability of MOOC platforms to provide automated feedback at scale. We explore features for every certified student's history of code submissions to specific problems in a programming MOOC and measure similarity to sample solutions. We seek to understand whether students who succeed in the course reach solutions similar to these instructor-intended sample solutions, in terms of the concepts and mechanisms they contain. Furthermore, do students learn to conform to instructor expectations as the course progresses, and does prior experience have correlations with student behavior? We also explore what feature representations are sufficient for code submission history, since they are directly applicable to the development of automated tutors for progress tracking.