The ever-increasing enrollments in programming courses has driven the need for sophisticated grading tools that can provide students with precise, insightful, and timely feedback. This SIGCSE workshop presents an interactive session on our powerful, open-source Python autograding framework, Pedal. As a free library, Pedal is available on a wide range of grading platforms, including GradeScope and BlockPy - anything that allows installation of a pure Python library. Pedal supports but goes beyond traditional unit testing, providing advanced code analysis techniques, such as type checking, liveness checking, structural code pattern matching, and more. Pedal has a large collection of assertions to evaluate dynamic program traces, query the Abstract Syntax Tree, re-execute student code under varying conditions, mock inputs, and capture outputs. Every feedback message is treated as a first-class object, empowering educators to fine-tune feedback as desired. Pedal is not just a toolset but a comprehensive pipeline optimized for feedback selection, resolution, and evaluation. With functionalities like command-line batch execution, exhaustive metadata tracking, and A/B testing, educators and researchers can analyze and refine their feedback strategies. With Pedal's successful deployment across multiple courses and institutions over the years, this workshop will offer attendees firsthand experience and a plethora of real-world examples. By the end of this workshop, participants will be proficient in leveraging Pedal, even venturing into the realm of creating interactive activities using the framework.

This demo introduces attendees to Pedal, a Python framework that streamlines the process of authoring semi-automated feedback on students? Python code. As a pure Python package, Pedal is compatible with a wide range of autograding platforms, including GradeScope, VPL, WebCAT, and BlockPy - as long as the platform allows package installation, Pedal should work. Pedal is a collection of modular program analysis tools exposed with a declarative interface, built around a centralized infrastructure. These tools include a sandboxed execution environment for running students' code with enhanced tracebacks, pattern matching syntax for specifying common student mistakes, basic type inference and flow analysis, random question pools, and a library of over 60 high-level, pedagogically-oriented assertions. Pedal's model for these tools synthesizes the detection of conditions and their instructor-mediated responses, encapsulated into dedicated feedback functions that can be tracked and modified as first-class objects. Our goal is to elevate Feedback with Software Engineering and Instructional Design practices, to become a central part of your course's development rather than an afterthought. Our toolchain also includes command lines utilities for unit testing your feedback to verify behavior and analyze collected programming snapshot data. Our hope is that adoptees will find that Pedal expands the power of their autograder and opens new avenues of research.

Pattern-based feedback detects incorrect code patterns in students' programs and provides feedback that can be personalized to the details of the matched code. Currently, a high level of instructor effort is required because the pattern detection must be expressed using complex programmatic interfaces. A specification language for pattern-based feedback is presented that mitigates this cost. Examples from actual student code illustrate the language's design and expressiveness. The language's implementation and testing is briefly described. Reflections are given on the the language design, where it is effectively used, and lessons learned from experience with its use. While our implementation is targeted at Python, other programming languages could be targeted using a similar approach.

In this paper, we introduce ProgSnap2, a standardized format for logging programming process data. ProgSnap2 is a tool for computing education researchers, with the goal of enabling collaboration by helping them to collect and share data, analysis code, and data-driven tools to support students. We give an overview of the format, including how events, event attributes, metadata, code snapshots and external resources are represented. We also present a case study to evaluate how ProgSnap2 can facilitate collaborative research. We investigated three metrics designed to quantify students' difficulty with compiler errors - the Error Quotient, Repeated Error Density and Watwin score - and compared their distributions and ability to predict students' performance. We analyzed five different ProgSnap2 datasets, spanning a variety of contexts and programming languages. We found that each error metric is mildly predictive of students' performance. We reflect on how the common data format allowed us to more easily investigate our research questions.

This paper describes Pedal, an innovative approach to the automated creation of feedback given to students in programming classes. Pedal is so named because it supports the PEDAgogical goals of instructors and is an expandable Library of components motivated by these goals. Pedal currently comes with components for type inferencing, flow analysis, pattern matching, and unit testing to provide an instructor with a rich set of resources to use in authoring and prioritizing feedback. The larger vision is the loosely-coupled architecture whose components can be readily expanded or replaced. The Pedal library components are motivated by a study of contemporary automated feedback systems and our own experience. Pedal's components are described and examples are given of Pedal-based feedback from three different introductory classes at two different universities. The integration of Pedal into several programming and autograding environments is briefly described.

Getting novice programmers over initial misconceptions is difficult because learning programming is difficult. Practice is one of the best ways for novices to learn. However, in the absence of feedback contextualized to instruction and focused on misconceptions, misconceptions become a difficult hurdle. To improve feedback, I present the Misconception-Driven Student Model (MDSM). MDSM is a cognitive model that lends itself to a framework to scalably deliver Misconception-Driven Feedback (MDF). I show MDF's impact through a quasi-experimental study that indicates that MDF significantly supports programming skill development. I plan on verifying these results by running another experimental study.

The feedback given to novice programmers can be substantially improved by delivering advice focused on learners' cognitive misconceptions contextualized to the instruction. Building on this idea, we present Misconception-Driven Feedback (MDF); MDF uses a cognitive student model and program analysis to detect mistakes and uncover underlying misconceptions. To evaluate the impact of MDF on student learning, we performed a quasi-experimental study of novice programmers that compares conventional run-time and output check feedback against MDF over three semesters. Inferential statistics indicates MDF supports significantly accelerated acquisition of conceptual knowledge and practical programming skills. Additionally, we present descriptive analysis from the study indicating the MDF student model allows for complex analysis of student mistakes and misconceptions that can suggest improvements to the feedback, the instruction, and to specific students.

Automated feedback needs instructor input to be most effective. The increasing demand in computing education necessitates automated feedback systems for teaching programming. However, most current automated feedback tools do not incorporate instructor input. Great strides are being made with identification and code edit steps for automated student feedback, but tools for instructor crafted feedback are lacking in the field of computing. My research, currently targeted at novice programmers aims to close that gap with a hybrid approach of a teacher in the loop feedback system I facilitate writing instructor feedback delivered to students in an automated fashion to give meaningful, instruction enhancing feedback. I also evaluate these mechanisms in classrooms by measuring learning gains, student perception, and other metrics.

This paper reports on a systematic method used to improve an existing unit of instruction. The method is distinctive in combining steps of instructional design with "knowledge components" from a cognitively-based framework of learning. Instructional design is used to develop assessment instruments that incorporate information about student misconceptions. The method uses the assessment instruments to evaluate student performance and learning gains, while statistical analysis evaluates the quality of the instruments themselves using measures of difficulty and discrimination. Fine-grain insight into possible improvements is enabled by the knowledge components implicated by the assessment. The method is illustrated and evaluated by applying it to a unit of instruction on collection-based iteration in a computational thinking class. Data gathered during this evaluation highlights a number of opportunities within the unit to refine the instruction.

This paper investigates the effects of physical objects as support for imagination in the context of enactive storytelling. More specifically, we target nine-year-old children because of their general disengagement from creative activity, a phenomenon known as the Fourth-grade Slump that arises from a demotivational spiral brought on by social awareness. We study how enactment using physical objects may allow the child to better engage in story imagination. Our study compares the richness of the imagination under three main enactment conditions with objects that have varying degrees of fidelity to referent objects: Cultural objects (physical visual resemblance); Physical objects (similar physical affordances); Arbitrary objects (minimal physical and visual affordances). We employ a mixed-methods analysis to gauge the child's level of broader imagination from three data sources: Enactment videos, drawings and interviews with the children. We found that the object types significantly differ in their support of the imagination, with the object of highest specificity being most effective. Our findings can inform the design of embodied creativity-support systems for children.