Prior research has established the importance of student instructor preferences and identified various influencing factors. However, the dynamics of how student instructor preferences develop and change are less well understood, due to the limitations of common course structures and reliance on one-time measurements. To bridge this gap, we utilize data from a novel learning platform that provides students with access to instructional content created by multiple instructors. This platform enables the quantification of preference emergence and evolution throughout an entire semester, as students repeatedly select content from different instructors. Examining both initial and final student instructor preferences suggests that preference is a dynamic construct continually shaped by experiences. Furthermore, our analysis of the associations between preferences and student characteristics reveals a nuanced picture: while student attributes did not significantly correlate with initial preferences, substantial differences emerged in final preferences across genders and self-reported prior programming experience. This analysis contributes to the existing body of knowledge by expanding our understanding of student instructor preferences and student-instructor relationships in computer science education. We also provide practical insights that institutions and instructors can draw on when multiple instructors collaborate on a course.

Debugging is a distinct subject in programming that is both comprehensive and challenging for novice programmers. However, instructors have limited opportunities to gain insights into the difficulties students encountered in isolated debugging processes. While qualitative studies have identified debugging strategies that novice programmers use and how they relate to theoretical debugging frameworks, limited larger scale quantitative analyses have been conducted to investigate how students’ debugging behaviors observed in log data align with the identified strategies and how they relate to successful debugging. In this study, we used submission log data to understand how the existing debugging strategies are employed by students in an introductory CS course when solving homework problems. We identified strategies from existing debugging literature that can be observed with trace data and extracted features to reveal how efficient debugging is associated with debugging strategy usage. Our findings both align with and contradict past assumptions from previous studies by suggesting that minor code edition can be a beneficial strategy and that width and depth aggregations of the same debugging behavior can reveal opposite effects on debugging efficiency.

Students in computerized learning environments often direct their own learning processes, which requires metacognitive awareness of what should be learned next. We investigated a novel method of measuring verbalized metacognition by applying natural language processing (NLP) to transcripts of interviews conducted in a classroom with 99 middle school students who were using a computerized learning environment. We iteratively adapted the NLP method for the linguistic characteristics of these interviews, then applied it to study three research questions regarding the relationships between verbalized metacognition and measures of 1) learning, 2) confusion, and 3) metacognitive problem-solving strategies. Verbalized metacognition was not directly related to learning, but was related to confusion and metacognitive problem-solving strategies. Results also suggested that interviews themselves may improve learning by encouraging metacognition. We discuss implications for designing computerized environments that support self-regulated learning through metacognition.

Confusion has been shown to be prevalent during complex learning and has mixed effects on learning. Whether confusion facilitates or hampers learning may depend on whether it is resolved or not. Confusion resolution, behind which is the resolution of cognitive disequilibrium, requires learners to possess some skills, but it is unclear what these skills are. One possibility may be metacognitive strategies (MS), strategies for regulating cognition. This study examined the relationship between confusion and actions related to MS in Betty's Brain, a computer-based learning environment. The results revealed that MS behavior differed during and outside confusion. However, confusion resolution was not related to MS behavior, and MS did not moderate the effect of confusion on learning.

Education research has explored the role of students' affective states in learning, but some evidence suggests that existing models may not fully capture the meaning or frequency of how students transition between different states. In this study we examine the patterns of educationally-relevant affective states within the context of Betty's Brain, an open-ended, computer-based learning system used to teach complex scientific processes. We examine three types of affective transitions based on similarity with the theorized D'Mello and Graesser model, transition between two affective states, and the sustained instances of certain states. We correlate of the frequency of these patterns with learning outcomes and our findings suggest that boredom is a powerful indicator of students' knowledge, but not necessarily indicative of learning. We discuss our findings within the context of both research and theory on affect dynamics and the implications for pedagogical and system design.

The relationship between learners' cognitive and affective states has become a topic of increased interest, especially because it is an important component of self-regulated learning (SRL) processes. This paper studies sixth grade students' SRL processes as they work in Betty's Brain, an agent-based open-ended learning environment (OELE). In this environment, students learn science topics by building causal models. Our analyses combine observational data on student affect to log files of students' interactions within the OELE. Preliminary analyses show that two relatively infrequent affective states, boredom and delight, show especially marked differences among high and low performing students. Further analysis shows that many of these differences occur after receiving feedback from the virtual agents in the Betty's Brain environment. We discuss the implications of these differences and how they can be used to construct adaptive personalized scaffolds.

MOOC research is typically limited to evaluations of learner behavior in the context of the learning environment. However, some research has begun to recognize that the impact of MOOCs may extend beyond the confines of the course platform or conclusion of the course time limit. This workshop aims to encourage our community of learning analytics researchers to examine the relationship between performance and engagement within the course and learner behavior and development beyond the course. This workshop intends to build awareness in the community regarding the importance of research measuring multi-platform activity and long-term success after taking a MOOC. We hope to build the community's understanding of what it takes to operationalize MOOC learner success in a novel context by employing data traces across the social web.

Completion rates for massive open online classes (MOOCs) are notoriously low. Identifying student patterns related to course completion may help to develop interventions that can improve retention and learning outcomes in MOOCs. Previous research predicting MOOC completion has focused on click-stream data, student demographics, and natural language processing (NLP) analyses. However, most of these analyses have not taken full advantage of the multiple types of data available. This study combines click-stream data and NLP approaches to examine if students' on-line activity and the language they produce in the online discussion forum is predictive of successful class completion. We study this analysis in the context of a subsample of 320 students who completed at least one graded assignment and produced at least 50 words in discussion forums, in a MOOC on educational data mining. The findings indicate that a mix of click-stream data and NLP indices can predict with substantial accuracy (78%) whether students complete the MOOC. This predictive power suggests that student interaction data and language data within a MOOC can help us both to understand student retention in MOOCs and to develop automated signals of student success.

This paper explores a longitudinal approach to combining engagement, performance and social connectivity data from a MOOC using the framework of exponential random graph models (ERGMs). The idea is to model the social network in the discussion forum in a given week not only using performance (assignment scores) and overall engagement (lecture and discussion views) covariates within that week, but also on the same person-level covariates from adjacent previous and subsequent weeks. We find that over all eight weekly sessions, the social networks constructed from the forum interactions are relatively sparse and lack the tendency for preferential attachment. By analyzing data from the second week, we also find that individuals with higher performance scores from current, previous, and future weeks tend to be more connected in the social network. Engagement with lectures had significant but sometimes puzzling effects on social connectivity. However, the relationships between social connectivity, performance, and engagement weakened over time, and results were not stable across weeks.

We investigated the interplay between confusion and in-game behavior among students using Newton’s Playground (NP), a computer game for physics. We gathered data from 48 public high school students in the Philippines. Upon analyzing quantitative field observations and interaction logs generated by NP, we found that confusion among students was negatively correlated with earning a gold badge (solving a problem with objects under par), positively correlated with earning a silver badge (solving a problem with objects over par), and positively correlated with stacking (drawing numerous small objects to reach the objective), a form of gaming the system.

Undergraduate STEM student enrollment has declined substantially over the last decade. Specifically there has been a steady decline in retention of early engineering students working through the first half of their degree programs. Student “leavers” typically fall into two categories (i) those facing academic difficulties and (ii) those that perceive the education environment of early engineering as hostile and not engaging. The Engaging Early Engineering Students Project (EEES) is a collaborative effort between Michigan State University (MSU) and Lansing Community College (LCC). EEES functions through the integration of four component programs designed to ease the transition of high school students into engineering undergraduate programs, and, by making the transition smoother, to increase retention at the College of Engineering (COE). The programs are: (a) Peer-Assisted Learning, (b) Connector Faculty, (c) Diagnostic-driven Early Intervention and (d) Cross Course linkages.

To reduce the prohibitive efforts associated to problem-solving ITS, a domain-independent framework can be developed. In this paper, we compare ASTUS to the MTT architecture with examples drawn from a scatter plot tutor. We show that both approaches are similar, but that an ASTUS tutor can exhibit a more sophisticated tutoring behavior thanks to its “pedagogical knowledge model”.