Programming courses can be challenging for first year university students, especially for those without prior coding experience.Students initially struggle with code syntax, but as more advanced topics are introduced across a semester, the difficulty in learning to program shifts to learning computational thinking (e.g., debugging strategies).This study examined the relationships between students' rate of programming errors and their grades on two exams.Using an online integrated development environment, data were collected from 280 students in a Java programming course.The course had two parts.The first focused on introductory procedural programming and culminated with exam 1, while the second part covered more complex topics and object-oriented programming and ended with exam 2. To measure students' programming abilities, 51095 code snapshots were collected from students while they completed assignments that were autograded based on unit tests.Compiler and runtime errors were extracted from the snapshots, and three measures -Error Count, Error Quotient and Repeated Error Density -were explored to identify the best measure explaining variability in exam grades.Models utilizing Error Quotient outperformed the models using the other two measures, in terms of the explained variability in grades and Bayesian Information Criterion.Compiler errors were significant predictors of exam 1 grades but not exam 2 grades; only runtime errors significantly predicted exam 2 grades.The findings indicate that leveraging Error Quotient with multiple error types (compiler and runtime) may be a better measure of students' introductory programming abilities, though still not explaining most of the observed variability.

A student’s ability to accurately evaluate the quality of their work holds significant implications for their self-regulated learning and problem-solving proficiency in introductory programming. A widespread cognitive bias that frequently impedes accurate self- assessment is overconfidence, which often stems from a misjudgment of contextual and task-related cues, including students’ judgment of their peers’ competencies. Little research has explored the role of overconfidence on novice programmers’ ability to accurately monitor their own work in comparison to their peers’ work and its impact on performance in introductory programming courses. The present study examined whether novice programmers exhibited a common cognitive bias called the "hard-easy effect", where students believe their work is better than their peers on easier tasks (overplace) but worse than their peers on harder tasks (underplace). Results showed a reversal of the hard-easy effect, where novices tended to overplace themselves on harder tasks, yet underplace themselves on easier ones. Remarkably, underplacers performed better on an exam compared to overplacers. These findings advance our understanding of relationships between the hard-easy effect, monitoring accuracy across multiple tasks, and grades within introductory programming. Implications of this study can be used to guide instructional decision making and design to improve novices’ metacognitive awareness and performance in introductory programming courses.

When students procrastinate on programming assignments, it can hinder the quality of their code and negatively impact their grades. In contrast, when students actively delay working on assignments to prepare to code (e.g., reading or seeking help), it can be an effective self-regulated learning (SRL) strategy beneficial to programming performance. However, distinguishing active delay from procrastination is methodologically challenging. To address this, we tracked what students did when they behaviorally delayed starting an assignment. Most students prepared to code by using multiple course resources across programming assignments. We found that many students delayed starting to code by seeking help in the Q&A platform, and this was beneficial to the quality of their code. Also, some pre-coding activities were related to behavioral delay in starting to code, but benefitted students' grades, and thus may indicate active delay, but not all pre-coding activities were beneficial. By considering pre-coding activities, we gain a comprehensive view of students' approach to coding in CS education.

Computing education researchers often study the impact of online help-seeking behaviors that occur across multiple online resources in isolation. Such separation fails to capture the interconnected nature of online help-seeking behaviors that occur across multiple online resources and its affect on course grades. This is particularly important for programming education, which arguably has more online resources to seek help from other people (e.g., computer-mediated conversations) than other majors. Using data from an introductory programming course (CS1) at a large US university, we found that students (n=301) sought help in multiple computer-mediated conversations, both Q&A forum and online office hours (OHQ), differently. Results showed the more prior knowledge about programming students had, the more they sought help in the Q&A compared to students with less prior knowledge. In general, higher-performing students sought help online in the Q&A more than the lower-performing groups on all the homework assignments, but not for the OHQ. By better understanding how students seek help online across multiple modalities of computer-mediated conversations and the relationship between help-seeking and grades, we can re-design online resources that best support all students in introductory programming courses at scale.

The effort to learn and the regulation of learning are key to successful learning. Voluntary practice has been shown to improve learning and is associated with having generally good self-regulated learning. At the same time, procrastination often slows the learning process and is associated with less than ideal regulation of learning. In this paper, we present the results of a study exploring the impact of voluntary practice and procrastination on the learning outcomes of novice programmers. We used data from an introductory programming course (CS1) at a large university and found that most students engaged in voluntary practice. However, students with higher prior performance and non-procrastinators were more likely to participate in the voluntary practice. We also found that participating in the voluntary practice did not have a significant impact on course performance. Furthermore, the study showed a weak negative correlation between procrastination and time spent on the homework and a weak negative correlation between procrastination and distributed practice. Finally, we found that non-procrastinators performed significantly better than procrastinators on the majority of homeworks.

In this article, we present the results of a study aiming to understand the impact of email nudge notification on students’ procrastination in office hours, and Piazza (QA forum) in a CS1 course at a large research university. With this study, we sought to understand if email nudges can be a useful tool in improving student’s learning behaviors, especially procrastination. After the first two homeworks, we randomly split students into two groups; the treatment group received the email, and the control group did not. The treatment group was further divided into two groups: one for the students who performed above the median (of the combined grades of homework 1 and 2) and those who performed below the median. Each sub-group received a slightly different version of the email. We found that students in the treatment group did not change their office hours’ attendance and Piazza interactions. We also found no difference in homework grades. However, students in the treatment group used more free late days on the following (third) homework. However, the change was short-lived, and they reverted to the pre-email level of late days usage on the fourth homework.

Vast numbers of publications in computing education begin with the premise that programming is hard to learn and hard to teach. Many papers note that failure rates in computing courses, and particularly in introductory programming courses, are higher than their institutions would like. Two highly distinct research projects have established that average success rates in introductory programming courses world-wide are in the region of 67%. However, there is little published work comparing pass rates in computing courses with those in other STEM disciplines. As institutions continually ask computing educators to justify the atypical failure rates in their courses, a thoroughly researched comparison of this sort could prove useful in demonstrating whether the phenomenon is real, and, if so, whether it extends somewhat beyond the boundaries of individual institutions. This working group will gather information on pass rates in computing courses, particularly introductory programming courses, and in courses at comparable levels in other STEM disciplines. Members of the group will be required to gather the information from their own institutions, and further data will be gathered by way of a broad survey. The data will be analysed to see whether global patterns can be established, and the group will survey the literature to gather and summarise postulated explanations for any difference between pass rates in computing and in other STEM disciplines.

Data Structures and Algorithms (DSA) courses are considered critical in any computer science curriculum. DSA courses emphasize topics related to procedural dynamics (how an algorithm works) and algorithm analysis (the algorithm's efficiency). Historically, algorithm visualizations (AVs) have dealt almost exclusively with portraying algorithm dynamics, and there are few examples of visualizations related to algorithm analysis topics. We have developed a new generation of visualizations that we term Algorithm Analysis Visualizations (AAVs) to convey algorithm analysis concepts. We present the motivation behind AAVs, and outlines a methodology for their evaluation. We present results from student surveys and the analysis of student interaction logs from the OpenDSA eTextbook used by several CS3-level classes during the period of Fall 2014 through Fall 2015. Initial results from Fall 2014 revealed that students were not spending enough time reading the algorithm analysis material presented as textual content. Our results from a preliminary deployment of AAVs in Spring 2015 showed that students interacted with AAVs for significantly longer than the control group spent reading the previous text-based algorithm analysis material. We will present additional results from our ongoing experiment in Fall2015 (control group without AAVs) and Spring2016 (test group with additional AAVs).

As the use of online interactive tutorials becomes more widespread, there will be more opportunities to use fine-grained interaction log data to deduce student behavior. Log data can help debug usability or pedagogical problems with the tutorials, or guide redesign to discourage pedagogically poor student behavior. OpenDSA is a collection of open source interactive materials for teaching data structures and algorithms. We present a case study analysis of the activity logs from use of OpenDSA tutorials by roughly 150 students over several weeks. We identified clusters of student use based on when they completed exercises, verified the reliability of estimated time requirements for exercises, provided evidence that a majority of students do not read the text, and found evidence that students complete additional exercises after obtaining credit. Furthermore, we determined that slideshow use was fairly high, but that skipping to the end of slideshows was common.

We present a study to evaluate OpenDSA, an open source, online system combining textbook-quality content with algorithm visualizations and interactive exercises for data structures and algorithms courses. We hypothesize that answering many questions and exercises with immediate feedback allows students to know whether they are on track with their learning. In a quasi-experimental study, a control group received lecture and textbook for three weeks. The treatment section spent class time working through equivalent content and exercises in OpenDSA. A post-test compared the two. An opinion survey examined students' perception and opinions about the experience. Detailed interaction logs were used to analyze student use of the tutorials and exercises to understand how they used the system.

OpenDSA is an open-source, community-based effort to create a complete active-eBook for Data Structures and Algorithms courses at the undergraduate level. Active-eBooks go beyond hypertextbooks, being a close integration of text and images with interactive visualizations and assessment activities. They solve two major problems: The difficulty of conveying dynamic process with static media, and the need by students to have many practice exercises and immediate feedback. Development in HTML5/JavaScript allows maximum portability. OpenDSA will proceed with broad participation from the algorithm visualization community. Focusing on reuse of materials, instructors can pick and choose content and modify as desired.