Background and Context. Problem decomposition is a core concern of computing education. It has also become increasingly relevant: in response to GenAI, many CS1 educators are advocating for shifting instructional emphasis away from code writing and towards decomposition and higher-level planning. Currently, there is a lack of knowledge in how novices do decomposition in large, multifunction tasks.

When engaged in the initial design of a program, novice programmers and seasoned developers alike often sketch out---or, perhaps more famously, whiteboard---their ideas. However, with the introduction of natively multimodal Generative AI models, such diagrams may now function as a means of code generation in their own right. In this work, we perform an initial evaluation to understand how student-created decomposition diagrams can serve as prompts for code generation, with implications for teaching and assessing problem decomposition skills.

Motivation: Q&A forums are a critical resource for supporting students in large educational environments, yet students often perceive these forums as stressful and report discomfort in participating visibly, especially in classes that are large and have students with varying levels of prior programming experience (PE). Method: We divided students in a CS1 Q&A forum into smaller, homogenous groups based on their PE. We use a mixed-methods approach to compare data from this experience to data from a setting where all students shared a single, large Q&A forum (a “mixed” setting). We quantitatively analyze measures of student engagement and use an open-ended qualitative approach to examine responses about student experience on the forums. This approach helps us identify the motivation behind student decisions to participate in visible or non-visible ways and to evaluate their alignment with theoretical frameworks. Results: In the mixed setting, students frequently use anonymity, with students without PE using anonymity more than students with PE and women using anonymity more than men. In contrast, in the homogenous groups, novices used anonymity less than novices in the mixed setting, while the students in higher-experience groups tended to use it more. We also observe a reduced anonymity usage among women in the homogenous experience groups, suggesting that PE plays a critical role in the observed gender disparities in forum participation. The qualitative analysis provides additional evidence that social status issues and confidence may explain these behavioral patterns. Conclusion: This study highlights the potential benefits and consequences of grouping students by experience. Homogenous PE groups foster increased student comfort and engagement within the Q&A forum for students with less experience, but students with more experience are exposed to more perceived status threats. We discuss how these results align with the theories we used to design the homogenous group setting. This exploration contributes to a deeper understanding of the underlying dynamics shaping student behavior in online learning communities. Educators and platform designers can use these lessons to more effectively create inclusive environments that accommodate diverse student needs and preferences.

Generative AI (GenAI) has seen great advancements in the past two years and the conversation around adoption is increasing. Widely available GenAI tools are disrupting classroom practices as they can write and explain code with minimal student prompting. While most acknowledge that there is no way to stop students from using such tools, a consensus has yet to form on how students should use them if they choose to do so. At the same time, researchers have begun to introduce new pedagogical tools that integrate GenAI into computing curricula. These new tools offer students personalized help or attempt to teach prompting skills without undercutting code comprehension. This working group aims to detail the current landscape of education-focused GenAI tools and teaching approaches, present gaps where new tools or approaches could appear, identify good practice-examples, and provide a guide for instructors to utilize GenAI as they continue to adapt to this new era.

The recent, widespread availability of Large Language Models (LLMs) like ChatGPT and GitHub Copilot may impact introductory programming courses (CS1) both in terms of what should be taught and how to teach it. Indeed, recent research has shown that LLMs are capable of solving the majority of the assignments and exams we previously used in CS1. In addition, professional software engineers are often using these tools, raising the question of whether we should be training our students in their use as well. This experience report describes a CS1 course at a large research-intensive university that fully embraces the use of LLMs from the beginning of the course. To incorporate the LLMs, the course was intentionally altered to reduce emphasis on syntax and writing code from scratch. Instead, the course now emphasizes skills needed to successfully produce software with an LLM. This includes explaining code, testing code, and decomposing large problems into small functions that are solvable by an LLM. In addition to frequent, formative assessments of these skills, students were given three large, open-ended projects in three separate domains (data science, image processing, and game design) that allowed them to showcase their creativity in topics of their choosing. In an end-of-term survey, students reported that they appreciated learning with the assistance of the LLM and that they interacted with the LLM in a variety of ways when writing code. We provide lessons learned for instructors who may wish to incorporate LLMs into their course.

A Concept Inventory (CI) is an assessment to measure student conceptual understanding of a particular topic. This article presents the results of a CI for basic data structures (BDSI) that has been previously shown to have strong evidence for validity. The goal of this work is to help researchers or instructors who administer the BDSI in their own courses to better understand their results. In support of this goal, we discuss our findings for each question of the CI using data gathered from 1,963 students across seven institutions.

Concept inventories (CIs) allow researchers and practitioners to measure student conceptual learning within a course or topic area. While they have enabled meaningful pedagogical change in other disciplines, there are relatively few CIs in computer science. In this paper, we report on our experiences as recent developers of a CI for basic data structures. We discuss each step along the route to a CI and offer tips based on what we have learned. We encourage others to create CIs, and we hope that this paper will serve as a practical guide through the process.

A Concept Inventory (CI) is a validated assessment to measure student conceptual understanding of a particular topic. This work presents a CI for Basic Data Structures (BDSI) and the process by which the CI was designed and validated. We discuss: 1) the collection of faculty opinions from diverse institutions on what belongs on the instrument, 2) a series of interviews with students to identify their conceptions and misconceptions of the content, 3) an iterative design process of developing draft questions, conducting interviews with students to ensure the questions on the instrument are interpreted properly, and collecting faculty feedback on the questions themselves, and 4) a statistical evaluation of final versions of the instrument to ensure its internal validity. We also provide initial results from pilot runs of the CI.

Although recent work in computing has discovered multiple techniques to identify low-performing students in a course, it is unclear what factors contribute to those students' difficulties. If we were able to better understand the characteristics of such students, we may be better able to help those students. This work examines the characteristics of low- and high-performing students through interviews with students from an introductory computing class. We identify a number of relevant areas of student behavior including how they approach their exam studies, how they approach completing programming assignments, whether they sought help after identifying misunderstandings, how and from whom they sought help, and how they reflected on assignments after submitting them. Particular behaviors within each area are coded and differences between groups of students are identified.

A number of recent studies in computer science education have explored the value of various data sources for early prediction of students' overall course performance. These data sources include responses to clicker questions, prerequisite knowledge, instrumented student IDEs, quizzes, and assignments. However, these data sources are often examined in isolation or in a single course. Which data sources are most valuable, and does course context matter? To answer these questions, this study collected student grades on prerequisite courses, Peer Instruction clicker responses, online quizzes, and assignments, from five courses (over 1000 students) across the CS curriculum at two institutions. A trend emerges suggesting that for upper-division courses, prerequisite grades are most predictive; for introductory programming courses, where no prerequisite grades were available, clicker responses were the most predictive. In concert, prerequisites and clicker responses generally provide highly accurate predictions early in the term, with assignments and online quizzes sometimes providing incremental improvements. Implications of these results for both researchers and practitioners are discussed.

As enrollments and class sizes in postsecondary institutions have increased, instructors have sought automated and lightweight means to identify students who are at risk of performing poorly in a course. This identification must be performed early enough in the term to allow instructors to assist those students before they fall irreparably behind. This study describes a modeling methodology that predicts student final exam scores in the third week of the term by using the clicker data that is automatically collected for instructors when they employ the Peer Instruction pedagogy. The modeling technique uses a support vector machine binary classifier, trained on one term of a course, to predict outcomes in the subsequent term. We applied this modeling technique to five different courses across the computer science curriculum, taught by three different instructors at two different institutions. Our modeling approach includes a set of strengths not seen wholesale in prior work, while maintaining competitive levels of accuracy with that work. These strengths include using a lightweight source of student data, affording early detection of struggling students, and predicting outcomes across terms in a natural setting (different final exams, minor changes to course content), across multiple courses in a curriculum, and across multiple institutions.

To be effective instructors and CS education researchers, we must identify and understand student difficulties surrounding core computing topics. This study examines student difficulties with the basic data structures commonly found in CS2 courses. Initial exploration of student thinking began with think-aloud interviews with students. These interviews centered on open-ended questions that were iteratively improved upon based on analysis of interview transcripts. The revised open-ended questions were then posed to 249 students during an end-of-term final exam study session. Using the explanations and justifications included by students, responses to the questions were coded and summarized. This work characterizes the difficulties revealed by student responses, and provides details of their prevalence among the examined student population.

Computing educators have used code review to provide opportunities for students to engage with code artifacts and to learn about aspects of programming, such as design and style, that are difficult to appreciate when working individually. However, most implementations of code review have been implemented in relatively small courses. For the past two years, we have asked students in our 500-950 person first year courses to engage in code reviews of submissions to course assignments. This short paper describes our approach for implementing code reviews that scale to large first-year courses and describes a few lessons we have learned from deploying these learning opportunities in our courses.

Recursion and iteration are two key topics taught in introductory Computer Science. This is especially so for CS2 students, as CS2 is the course where recursion is typically taught and where control-flow concepts are solidified. When asked to solve a problem that could feasibly be solved with recursion or iteration, what do CS2 students choose to do? And how does this choice relate to the correctness of their code? This paper provides one answer to these questions through an analysis of student exam responses to a problem on finding deepest common ancestors in trees.

Establishing learning goals for a course allows instructors to design course content to address those goals, helps students to focus their learning appropriately, and enables researchers to assess learning of those goals. In this work, we propose six learning goals for a topic prevalent in CS2 courses: Basic Data Structures. These learning goals arise from reviewing several CS2 courses at a variety of institutions, surveying faculty experts who commonly teach CS2, and meeting and working closely with these experts. We outline our process for creating learning goals, identify important topics underlying these goals, and provide examples of how the goals developed on the path to consensus. We also document that the term "CS2" does not have a unified interpretation within the CS education community and describe how this hurdle influenced our decision to focus on Basic Data Structures.

Dynamic Programming (DP) is considered to be one of the most difficult topics for students to understand in theoretical CS. Prior work suggests that misconceptions arise even when students have completed a course in which there is considerable focus on learning how to solve DP problems. We conducted think-aloud interviews with students who have completed the DP portion of the Algorithms course at a top North American research university. We report on three themes and their misconceptions discovered through this process. The first theme delves into students' struggles defining the notion of a subproblem and identifying particular subproblems. The second theme focuses on the understanding and usage of DP solution techniques compared to other algorithmic approaches. The third theme is composed of misconceptions related to defining and using recurrences. Analysis of each misconception provides insight into student thinking and offers ideas for improving the education of DP to university students.

Replication research is rare in CS education. For this reason, it is often unclear to what extent our findings generalize beyond the context of their generation. The present paper is a replication and extension of Achievement Goal Theory research on CS1 students. Achievement goals are cognitive representations of desired competence (e.g., topic mastery, outperforming peers) in achievement settings, and can predict outcomes such as grades and interest. We study achievement goals and their effects on CS1 students at six institutions in four countries. Broad patterns are maintained --- mastery goals are beneficial while appearance goals are not --- but our data additionally admits fine-grained analyses that nuance these findings. In particular, students' motivations for goal pursuit can clarify relationships between performance goals and outcomes.

An exam wrapper is a structured activity that students engage in after their instructor has graded and returned an exam, and is designed to promote self-reflection and improve study practices. This paper describes two studies examining the efficacy and student perceptions of exam wrappers. The studies were conducted at two major Canadian universities, using complementary research designs. We report that neither study produced evidence that exam wrappers have a significant effect on final exam scores or on course drop rates. However, we also find that the use of wrappers was associated with improved rates of test pickup and increased scores on a course evaluation question regarding the fairness of evaluation methods. Given these results, we advise instructors who are considering the use of exam wrappers to review the evidence for other possible interventions that may more effectively serve the same goals.

With growing enrolment and ongoing research into best practices for team work, many instructors are rethinking how to form, evaluate, and manage teams. In this BoF, instructors will discuss the strategies they have employed, the effectiveness of those approaches, and the tools that support administering teams. Discussion topics may include self- and peer-evaluation, grading strategies, software support, and conflict resolution.

Although computing students may enjoy when their instructors teach using analogies, it is unknown to what extent these analogies are useful for their learning. This study examines the value of analogies when used to introduce three introductory computing topics. The value of these analogies may be evident during the teaching process itself (short term), in subsequent exams (long term), or in students' ability to apply their understanding to related non-technical areas (transfer). Comparing results between an experimental group (analogy) and control group (no analogy), we find potential value for analogies in short term learning. However, no solid evidence was found to support analogies as valuable for students in the long term or for knowledge transfer. Specific demographic groups were examined and promising preliminary findings are presented.

Being able to identify low-performing students early in the term may help instructors intervene or differently allocate course resources. Prior work in CS1 has demonstrated that clicker correctness in Peer Instruction courses correlates with exam outcomes and, separately, that machine learning models can be built based on early-term programming assessments. This work aims to combine the best elements of each of these approaches. We offer a methodology for creating models, based on in-class clicker questions, to predict cross-term student performance. In as early as week 3 in a 12-week CS1 course, this model is capable of correctly predicting students as being in danger of failing, or not, for 70% of the students, with only 17% of students misclassified as not at-risk when at-risk. Additional measures to ensure more broad applicability of the methodology, along with possible limitations, are explored.

The programming education literature includes many observations that pass rates are low in introductory programming courses, but few or no comparisons of student performance across courses. This paper addresses that shortcoming. Having included a small set of identical questions in the final examinations of a number of introductory programming courses, we illustrate the use of these questions to examine the relative performance of the students both across multiple institutions and within some institutions. We also use the questions to quantify the size and overall difficulty of each exam. We find substantial differences across the courses, and venture some possible explanations of the differences. We conclude by explaining the potential benefits to instructors of using the same questions in their own exams.

Ten selected questions have been included in 13 introductory programming exams at seven institutions in five countries. The students' results on these questions, and on the exams as a whole, lead to the development of a benchmark against which the exams in other introductory programming courses can be assessed. We illustrate some potential benefits of comparing exam performance against this benchmark, and show other uses to which it can be put, for example to assess the size and the overall difficulty of an exam. We invite others to apply the benchmark to their own courses and to share the results with us.

In addition to their role as a summative measure, midterm tests can provide formative feedback that can be used by students to identify areas of weakness and adjust studying approaches. Unfortunately, low levels of test pickup often preclude this type of learning from tests. Even when students do collect their marked tests, it is unclear how much they reflect on or learn from the feedback.

Peer Instruction (PI) is a student-centric pedagogy in which students move from the role of passive listeners to active participants in the classroom. Over the past five years, there have been a number of research articles regarding the value of PI in computer science. The present work adds to this body of knowledge by examining outcomes from seven introductory programming instructors: three novices to PI and four with a range of PI experience. Through common measurements of student perceptions, we provide evidence that introductory computing instructors can successfully implement PI in their classrooms. We find encouraging minimum (74%) and average (92%) levels of success as measured through student valuation of PI for their learning. This work also documents and hypothesizes reasons for comparatively poor survey results in one course, highlighting the importance of the choice of grading policy (participation vs. correctness) for new PI adopters.

No abstract available.

Achievement goals are cognitively-represented end states that individuals strive to reach in competence situations. Well-studied by educational psychologists, achievement goals are robust predictors of grades, interest, and motivation of students. In this paper, we apply achievement goal theory to measure CS1 students' achievement goals and consequent interest in CS and final exam grade. We find that students aiming for topic mastery become interested in CS and, contrary to theoretical expectations, perform well on the exam. A more complex pattern of results surrounds students who orient toward competence demonstration or normative comparison, and the link between such performance goals and outcomes is less clear. We argue for the continued appropriation of educational theory to inform our studies of CS success.

Recent work in computing has converged on a collection of complementary findings suggesting the value of the Peer Instruction (PI) pedagogy. Compared to lecture, PI has been shown to decrease fail rates, increase final exam grades, and increase engagement and enjoyment. In PI, students work together to exchange perspectives and use clickers to answer challenging conceptual questions in the presence of a knowledgeable instructor.

Recent computer science education research using isomorphic questions in Peer Instruction (PI) classes demonstrates that students learn from talking to their peers and listening to the instructor's follow-up explanation. These results provide evidence of the effectiveness of the PI process but are limited to what happens in a single class session. The present work extends the use of isomorphic questions to investigate how in-class learning translates to success on the final exam. Despite considerable time between in-class questions and the final exam, we find that students who learn in class are shown to retain that learning and to perform better than students who did not learn in class. In addition, compared to students already understanding the material, those who learned the material in class are almost (87%) as likely to correctly answer isomorphic exam questions. Our results have implications for the value of difficult PI questions and the meaning of in-class response graphs.

Recent work in computing has converged on a collection of complementary findings suggesting the value of the Peer Instruction (PI) pedagogy. Compared to lecture, PI has been shown to decrease fail rates, increase final exam grades, and increase engagement and enjoyment. In PI, students work together to exchange perspectives and use clickers to answer challenging conceptual questions in the presence of a knowledgeable instructor.

Computer Science 1 (CS1), the first course taken by college-level computer science (CS) majors, has traditionally suffered from high failure rates. Efforts to understand this phenomenon have considered a wide range of predictors of CS success, such as prior programming experience, math ability, learning style, and gender, with findings that are suggestive but inconclusive. The current quasiexperimental study extends this research by exploring how the pedagogical approach of the course (traditional lecture vs. Peer Instruction (PI) and clickers) in combination with student achievement goals (mastery goals vs. performance goals) relates to exam grades, interest in the subject matter, and course enjoyment. The research revealed that students with performance goals scored significantly lower on final exams in both the lecture and PI conditions. However, students with performance goals reported higher levels of subject matter interest when taught through PI. Students with mastery goals, in both conditions, scored significantly higher on the final exam, had higher levels of interest, and reported higher levels of course enjoyment than their performance-oriented counterparts. The results suggest that PI may improve the level of subject-matter interest for some students, thereby indicating the importance of studying pedagogical approach as we seek to understand student outcomes in CS1.

Recent research suggests that the first weeks of a CS1 course have a strong influence on end-of-course student performance. The present work aims to refine the understanding of this phenomenon by using in-class clicker questions as a source of student performance. Clicker questions generate per-lecture and per-question data with which to assess student understanding. This work demonstrates that clicker question performance early in the term predicts student outcomes at the end of the term. The predictive nature of these questions applies to code-writing questions, multiple choice questions, and the final exam as a whole. The most predictive clicker questions are identified and the relationships between these questions and final exam performance are examined.

We compare a traditional CS1 offering with an inverted offering delivered the following year to a comparable student population. We measure student attitudes, grades, and final course outcomes and find that, while students in the inverted offering do not report increased enjoyment and are no more likely to pass, learning as measured by final exam performance increases significantly. This increase is not simply a function of a more onerous inverted offering, as students report spending similar time per week in the traditional and inverted offerings. Contrary to our hypotheses, however, we find no evidence that the the inverted offering disproportionally helps beginners or those not fully fluent in English.

In computer science, the active learning pedagogical practice of Peer Instruction (PI) has been shown to improve final exam performance, reduce student failure rates, and improve student retention. PI consists of two major parts: group discussion and follow-up instructor intervention. We expect that PI performance as a whole will correlate with final exam performance, but it is unclear whether or how each piece of PI is involved in these relationships. In this work, we use isomorphic questions to isolate the effects of peer discussion and instructor intervention, and examine scores on a final exam and its code-writing and code-tracing questions. We find that both pieces of PI correlate with the final exam as a whole, code-tracing question (similar to PI questions), and code-writing question (not similar to PI questions). This is further evidence that both PI components are important to the success of PI.

We examine student difficulties with CS1 concepts by analyzing a dataset containing 266,852 student responses to weekly code-writing problems. We find that conditionals and loops prove particularly problematic, even when considering 'second chance' data; and that, while we observe some evidence of improvement, certain straightforward applications of loops continue to be problematic at the end of the term. Our contribution is the corroboration of earlier findings, and a call to use online repositories of student submissions as rich sources of data on the student learning experience.

Short, automatically-assessed programming exercises, and other types of short practice problems, are a useful way to introduce and reinforce concepts and techniques in introductory programming courses. When delivered over the web, they allow students to learn and practice, with immediate feedback, at any time and place where they have access to a web browser. However, such exercises do not seem to be as widely used as they could be. Similarly, there is not a lot of literature on the effectiveness of these types of problems. The purpose of this BOF is to bring together users (and potential users) of programming exercises with developers of programming exercise systems to discuss how exercises could be used more widely and effectively. Possible discussion topics include: What features are absolutely essential for faculty to consider adoption? What are the major obstacles preventing more widespread adoption? Are faculty willing to share their exercises under an open/non-commercial license? Should exercises best used for extra practice, as graded assignments, or both?

It is generally assumed that early success in CS1 is crucial for success on the exam and course as a whole. Particularities of students, densely-connected CS1 content, and recurring core topics each suggest that it is difficult to rebound from early misunderstandings. In this paper, we use Peer Instruction (PI) data, in addition to exam data, to explore relationships between in-class assessments and performance at the end of term and on the exam. We find that early course performance very quickly and strongly predicts performance on the final exam and that subsequent weeks provide no major increase in that predictive power. In contrast, early performance is similarly predictive of performance in the last weeks of PI questions, but subsequent weeks are increasingly more predictive. We speculate on what this means for the content of these assessments and potential future assessment practices.

Recent work in computing suggests that Peer Instruction (PI) is a valuable interactive learning pedagogy: it lowers fail rates, increases retention, and is enjoyed by students and instructors alike. While these findings are promising, they are somewhat incidental if our goal is to understand whether PI is "better" than lecture in terms of student outcomes. Only one recent study in computing has made such a comparison, finding that PI students outperform traditionally-taught students on a CS0 final exam. That work was conducted in a CS0, where the same instructor taught both courses, and where the only outcome measure was final exam grade. Here, I offer a study that complements their work in two ways. First, I argue for and measure self-efficacy as a valued outcome, in addition to that of final exam grade. Second, I offer an inter-instructor CS1 study, whose biases differ from those of intra-instructor studies. I find evidence that PI significantly increases self-efficacy and suggestively increases exam scores compared to a traditional lecture-based CS1 class. I note validity concerns of such an in-situ study and offer a synthesis of this work with the extant PI literature.

Many studies have demonstrated that students tend to learn less than instructors expect in CS1. In light of these studies, a natural question is: to what extent do these results hold for subsequent, upper-division computer science courses? In this paper we describe our work in creating high-level concept questions for an upper-division computer architecture course. The questions were designed and agreed upon by subject-matter and teaching experts to measure desired minimum proficiency of students post-course. These questions were administered to four separate computer architecture courses at two different institutions: a large public university and a small liberal arts college. Our results show that students in these courses were indeed not learning as much as the instructors expected, performing poorly overall: the per-question average was only 56%, with many questions showing no statistically significant improvement from pre-course to post-course. While these results follow the trend from CS1 courses, they are still somewhat surprising given that the courses studied were taught using research-based pedagogy that is known to be effective across the CS curriculum. We discuss implications of our findings and offer possible future directions of this work.

We present the Python Classroom Response System, a web-based tool that enables instructors to use code-writing and multiple choice questions in a classroom setting. The system is designed to extend the principles of peer instruction, an active learning technique built around discussion of multiple- choice questions, into the domain of introductory programming education. Code submissions are evaluated by a suite of tests designed to highlight common misconceptions, so the instructor receives real-time feedback as students submit code. The system also allows an instructor to pull specific submissions into an editor and visualizer for use as in-class examples. We motivate the use of this system, describe its support for and extension of peer instruction, and offer use cases and scenarios for classroom implementation.

We introduce participants to Peer Instruction (PI): an active learning technique shown to be effective across the CS curriculum. In PI, Students work together to exchange perspectives and answer challenging conceptual questions, and are supported by short teaching segments. We will introduce and motivate PI, demonstrate its use in combination with a clicker system, and describe ways to encourage student preparation for PI classes. Pre-lecture preparation may include reading quizzes or exploratory homeworks, two topics of recent interest in the computing research literature. We will explore this literature as part of our tour of a complete PI course: from pre-lecture, to lecture, to the course at large.

Peer Instruction has recently gained interest in computing as an effective active learning pedagogy. The general focus of PI research has been on the in-class portion of PI: multiple choice questions and group discussion. Here, our focus is the reading quizzes completed by students for purposes of class preparation. These quizzes contain content questions but also ask for difficulties or confusion with course material. Consistent with expectations, we demonstrate that providing correct responses to quiz questions positively correlates with other course assessments. Somewhat counter-intuitively, we find that identifying confusions, noting problematic sections, or asking questions about the reading are also correlated with lab grades.

No abstract available.

We introduce participants to Peer Instruction (PI): an active learning technique applicable to the teaching of many subjects, including CS. In PI, Students work together to exchange perspectives and answer challenging conceptual questions, and are supported by short teaching segments. We will introduce and motivate PI, demonstrate its use in combination with a clicker system, and show that PI is much more than the use of clickers. Participants will work in groups to develop new PI questions addressing challenges to their students' learning, and discuss numerous pedagogical benefits conferred through PI.

In this paper, we explore the use of sequences of small code writing questions ("concept questions") designed to incrementally evaluate single programming concepts. We report on a study of student performance on a CS1 final examination that included a traditional code-writing question and four intentionally corresponding concept questions. We find that the concept questions are significant predictors of performance on both the corresponding code-writing question and the final exam as a whole. We argue that concept questions provide more accurate formative feedback and simplify marking by reducing the number of variants that must be considered. An analysis of responses categorized by the students' previous programming experience suggests that inexperienced students have the most to gain from the use of concept questions.

Peer Instruction (PI) is an instructional approach that engages students in constructing their own understanding of concepts. Students individually respond to a question, discuss with peers, and respond to the same question again. In general, the peer discussion portion of PI leads to an increase in the number of students answering a question correctly. But are these students really learning, or are they just "copying" the right answer from someone in their group? In an article in the journal Science, Smith et al. affirm that genetics students individually learn from discussion: having discussed a first question with their peers, students are better able to correctly, individually answer a second, conceptually-related question. We replicate their study, finding that students in upper-division computing courses (architecture and theory of computation) also learn from peer discussions, and explore differences between our results and those of Smith et al. Our work reveals that using raw percentage gains between paired questions may not fully illuminate the value of peer discussion. We define a new metric, Weighted Learning Gain, which better reflects the learning value of discussion. By applying this metric to both genetics and computing courses, we consistently find that 85-89% of "potential learners" benefit from peer discussion.

Peer Instruction (PI) has a significant following in physics, biology, and chemistry education. Although many CS educators are aware of PI as a pedagogy, the adoption rate in CS is low. This paper reports on four instructors with varying motivations and course contexts and the value they found in adopting PI. Although there are many documented benefits of PI for students (e.g. increased learning), here we describe the experience of the instructor by looking in detail at one particular question they posed in class. Through discussion of the instructors' experiences in their classrooms, we support educators in consideration of whether they would like to have similar classroom experiences. Our primary findings show instructors appreciate that PI assists students in addressing course concepts at a deep level, assists instructors in dynamically adapting their class to address student misunderstandings and, overall, that PI encourages students to be engaged in conversations which help build technical communication skills. We propose that using PI to engage students in these activities can effectively support training in analysis and teamwork skills.

Many factors have been cited for poor performance of students in CS1. To investigate how assessment mechanisms may impact student performance, nine experienced CS1 instructors reviewed final examinations from a variety of North American institutions. The majority of the exams reviewed were composed predominantly of high-value, integrative code-writing questions, and the reviewers regularly underestimated the number of CS1 concepts required to answer these questions. An evaluation of the content and cognitive requirements of individual questions suggests that in order to succeed, students must internalize a large amount of CS1 content. This emphasizes the need for focused assessment techniques to provide students with the opportunity to demonstrate their knowledge.