ILDBug is a novel debugging approach inspired by the pedagogical technique Interactive Lecture Demonstrations (ILDs). During ILDs, students predict a demonstration's result, experience the demonstration, and then reflect on their experience. We adapted this process to teach debugging (ILDBug) by having students engage in detecting the bug for prediction, then locating and correcting the bug for experience, and then reflecting on the debugging process. The ILDBug approach is designed to be a lightweight technique to create a debugging exercise that is adaptable to many contexts. This demo will cover the high-level pieces the audience needs to create their own ILDBug using our approach, an example ILDBug in an introductory programming context so they can see it in action, and finally tips on how to adapt our technique to fit their own classroom context.

Instructors face the challenge of encouraging well-tested, quality code submissions from students while battling the double-edged sword of the autograder. While autograders can provide feedback to students quickly, students can become reliant on the autograder as the primary means for determining correctness of their code. In a similar spirit, instructors also frequently promote submitting early and not waiting until the last second. To encourage students to submit fewer erroneous submissions and completing programming assignments earlier, we examine two lab submission policies, time-restricted submissions and point-restricted submissions, implemented in consecutive semesters of a large Computer Architecture course. We survey students on their perception of these two policies and analyze the lab data and compare between the two semesters. Our initial results show that while students preferred the time-restricted policy, the point-restricted policy is positively affecting students' correct submission percentage, as well as students submitting correct submissions earlier.

This Research Full Paper presents an observation protocol to explore group processing in cooperative learning. Use of structured roles, such as Process Oriented Guided Inquiry Learning (POGIL) and pair programming, can help facilitate cooperative learning and help courses scale to large classroom sizes, decrease attrition and failure rates, and improve student performance. Our observation protocol was created to capture how groups work together in online, POGIL-inspired activities. A team of graduate student researchers developed the observation protocol for a variety of courses by observing three different computer science courses during the Spring 2021 semester. A total of 77 groups across all three courses were recorded, and percent agreement using a subset of the recordings suggested good interrater reliability (91.29%). We also extend a previous equality metric to quantify the rates of student participation, and found that it offered good differentiation between groups where one student contributed the most and groups where students contributed equally. We present example applications of our observation protocol related to general participation trends, the kinds of contributions students make, student-student bonding, and help-seeking patterns. Finally, we discuss future directions for use of our coding scheme as well as implications for implementing structured role-based cooperative learning online in the future.

To defend against collaborative cheating in code writing questions, instructors of courses with online, asynchronous exams can use the strategy of question variants. These question variants are manually written questions to be selected at random during exam time to assess the same learning goal. In order to create these variants, currently the instructors have to rely on intuition to accomplish the competing goals of ensuring that variants are different enough to defend against collaborative cheating, and yet similar enough where students are assessed fairly. In this paper, we propose data-driven investigation into these variants. We apply our data-driven investigation into a dataset of three midterm exams from a large introductory programming course. Our results show that (1) observable inequalities of student performance exist between variants and (2) these differences are not just limited to score. Our results also show that the information gathered from our data-driven investigation can be used to provide recommendations for improving design of future variants.

We analyze the submissions of 286 students as they solved Structured Query Language (SQL) homework assignments for an upper-level databases course. Databases and the ability to query them are becoming increasingly essential for not only computer scientists but also business professionals, scientists, and anyone who needs to make data-driven decisions. Despite the increasing importance of SQL and databases, little research has documented student difficulties in learning SQL. We replicate and extend prior studies of students' difficulties with learning SQL. Students worked on and submitted their homework through an online learning management system with support for autograding of code. Students received immediate feedback on the correctness of their solutions and had approximately a week to finish writing eight to ten queries. We categorized student submissions by the type of error, or lack thereof, that students made, and whether the student was eventually able to construct a correct query. Like prior work, we find that the majority of student mistakes are syntax errors. In contrast with the conclusions of prior work, we find that some students are never able to resolve these syntax errors to create valid queries. Additionally, we find that students struggle the most when they need to write SQL queries related to GROUP BY and correlated subqueries. We suggest implications for instruction and future research.

Enrollment in introductory programming (CS1) courses continues to surge and hundreds of CS1 students can produce thousands of submissions for a single problem, all requiring timely and accurate grading. One way that instructors can efficiently grade is to construct a custom instructor test suite that compares a student submission to a reference solution over randomly generated or hand-crafted inputs. However, such test suite is often insufficient, causing incorrect submissions to be marked as correct. To address this issue, we propose the Grasa (GRAding-based test Suite Augmentation) approach consisting of two techniques. Grasa first detects and clusters incorrect submissions by approximating their behavioral equivalence to each other. To augment the existing instructor test suite, Grasa generates a minimal set of additional tests that help detect the incorrect submissions. We evaluate our Grasa approach on a dataset of CS1 student submissions for three programming problems. Our preliminary results show that Grasa can effectively identify incorrect student submissions and minimally augment the instructor test suite.