Background and Context. The prominence of Generative AI (GenAI) has led students to seek academic support from these tools, altering the help-seeking landscape. Since GenAI can operate as an on-demand help resource, understanding the unique value of human staff can help instructors better serve students.

Background and Context. Problem decomposition is a core concern of computing education. It has also become increasingly relevant: in response to GenAI, many CS1 educators are advocating for shifting instructional emphasis away from code writing and towards decomposition and higher-level planning. Currently, there is a lack of knowledge in how novices do decomposition in large, multifunction tasks.

This is a place to foster a growing community of computing educators around teaching in prison – for those who are interested in possibly doing this in the future, and those with plans or experience doing so. Higher education in prison (HEP) has expanded rapidly in the U.S. over the past several years after a policy change that re-instated pell grant eligibility to incarcerated adults. This follows a global shift toward more rehabilitative, as opposed to punitive, strategies toward criminal justice as more countries recognize the wide-ranging benefits to all members of society. However, computing education, as well as basic digital literacy, remain a challenge as various factors like technology infrastructure, perceived threats to security, and instructor willingness remain challenges to offering CS courses in prison education programs. Despite these barriers, there are several models of CS education happening in prison today including in-person and virtual instruction, for-credit college courses and informal workshops. In this session, we will talk about different ways of getting started with teaching in prison, as well as practical strategies for navigating challenges from our own personal experience of teaching CS in prison settings.

This paper introduces the PrairieLearn Question Sharing System (PQSS), which enables instructors to share question generators with other instructors, either as open educational resources or privately. PQSS is integrated into PrairieLearn, an open-source, problem-driven online learning platform. PQSS addresses a critical need for more open-source assessments by making it easier for instructors to share assessments and for instructors to use those assessments. Instructors often do not share questions due to the time it takes to publish them and the lack of recognition for their work. Because it is directly integrated into PrairieLearn, PQSS reduces the aforementioned friction of sharing and using shared questions, and we can report usage statistics to help question authors receive recognition for their work. In this paper, we share design and implementation details of the system, as well as experiences using it to share course content across courses and between universities.

When engaged in the initial design of a program, novice programmers and seasoned developers alike often sketch out---or, perhaps more famously, whiteboard---their ideas. However, with the introduction of natively multimodal Generative AI models, such diagrams may now function as a means of code generation in their own right. In this work, we perform an initial evaluation to understand how student-created decomposition diagrams can serve as prompts for code generation, with implications for teaching and assessing problem decomposition skills.

Incarcerated students face profound barriers to learning computer science: restricted internet access, limited technology, and highly constrained peer interaction. In this environment, the learning management system (LMS) becomes not only a tool but a central site for teaching, collaboration, and meaning-making. This poster presents a quantitative LMS trace analysis of a Winter 2025 computer architecture course taught in a California prison, examining how teaching presence (professor/TA facilitation and response timing) related to discussion board activity, LMS page views, and grade outcomes through the lens of the Community of Inquiry (CoI) framework. Using Canvas API exports of discussion board content, page views, and grades, we analyzed participation counts by role, thread depth, staff response times, and correlations between engagement and final percentage. We extend prior work by providing quantitative and temporal visualizations of participation, summarizing posting patterns across the 10-week course. We also report preliminary indicators of message content characteristics and clarify our protocol for distinguishing staff and student contributions. While these analyses represent a single cohort, they offer early evidence that timely teaching presence may support engagement in constrained environments and suggest directions for broader application in online, hybrid, and other resource-limited CS courses.

Recent work has shown that incarcerated adult students reported a decrease in confidence in their ability to do well in the course as the course progressed, whereas non-incarcerated students in a traditional educational setting reported an increase in confidence over time on the same measure. Given these differences in student experiences between incarcerated adult students and traditional students, this work seeks to further understand the experiences of incarcerated adult in CS1, with a focus on their attitudes towards computing. Specifically, we used the Computing Attitude Survey (CAS) as a pre/post measurement in a CS1 course taught in prison. We found significant positive shifts for Problem Solving - Transfer and Fixed Mindset factors and a slight decrease for Real-World Connections. We additionally compare the results of the CAS survey between the incarcerated adult students in this study and those of non-incarcerated students reported in prior work.

Culturally Relevant Computing (CRC) has been shown to have positive effects on students, including increased classroom engagement, increased computing interest, and increased math performance. However, the vast majority of CRC studies take place in primary and secondary education, with little known about its efficacy in higher education. Despite the lack of literature on CRC in higher education, we believe that there may be CRC techniques presently being implemented in this context---yet not published in the research community. To investigate, we interviewed 21 professors from Hispanic Serving Institutions within the Computing Alliance for Hispanic Serving Institutions to document their implementation (if any) of CRC in the higher education context. Professors reported several CRC implementations including, but not limited to, culture in course materials, language-based approaches, providing opportunities outside the classroom, cultural sharing, and professional development. We conclude by discussing how these results may contribute to the promotion of leveraging students' cultures in higher education to serve students---particularly of minoritized backgrounds in computing.

In this discussion, we seek to create a space for computing educators interested in teaching in prisons. Recent policy changes are leading to a growing number of higher education institutions providing programs in prison to combat the national crisis of mass incarceration, which disproportionately impacts already marginalized groups. Within the computing education community, we see an opportunity for several mutual benefits through our increased involvement in these programs including: a) providing computing courses in prisons, where STEM education opportunities are severely lacking, b) increasing diverse perspectives by bringing justice-impacted people into computing, and c) gaining experience and improving known methods in effective CS education in settings with low-technology infrastructure, for adult and non-traditional students, and for students from diverse backgrounds. All discussion leaders have experience in and/or plans to teach CS in a college-in-prison program. No experience is required, and we welcome all members of the community interested in learning more about these opportunities.

Understanding incarcerated adult (IA) students' fears upon entering a CS1 course and how their confidence changes throughout the course can help us understand how well IA students' fears are being addressed, and help future instructors of CS1 in prison address them better. Building on recent work on non-CS majors' fears and confidence in introductory CS, we surveyed 45 IA students across two offerings of a CS1 course in prison on their fears going into the course, and confidence in their ability to succeed. We present a phenomenographic analysis of fears amongst IA students in CS1, and analyze relationships between these fears and change in confidence. In addition, we compare the fears expressed by IA students to those of non-CS majors from prior work. Findings include many IA students reporting no fears, but an overall decrease in confidence across both offerings of the course which was mostly accounted for by students who did express initial fears. We found 9 fears overlapping with those found in a prior study outside of the prison context (e.g., getting a bad grade), and 7 fears only identified in our study (e.g., interference from circumstances beyond my control).

Undergraduate teaching assistants (tutors) are commonly employed in computing courses to help students with programming assignments. Prior research in computing education has reported the benefits of tutoring both for students and for the tutors' own learning. In contrast, recent research that examined actual tutoring sessions has reported that these sessions may be less productive than one might hope, with tutors often just giving students the answers to their problems without trying to teach the underlying concepts. To better understand why tutors may be employing these suboptimal practices, we interviewed ten tutors across early computing courses in higher education to identify their perceived role in these sessions, what stressors and factors influence their ability to perform their job effectively, and what kinds of best practices they learned in their tutor training course. Tutors reported their roles around student learning, gauging student understanding, identifying or providing solutions to students, and providing socioemotional support. They reported their stressors around environmental factors (e.g., number of students waiting to be helped, preparation time, peer-tutor frustrations), internal influences, student behavior, student skill levels, and feeling the need to ''read a student's mind.'' Regarding their tutor training course, Tutors reported learning about interaction guidelines and procedures and question-based problem solving. We conclude by discussing how these results may contribute to the less-effective behaviors seen in prior research and potential ways to improve tutoring in computing courses.

Large language models (LLMs) are already heavily used by professional software engineers. An important skill for new university graduates to possess will be the ability to use such LLMs to effectively navigate and modify a large code base. While much of the prior work related to LLMs in computing education focuses on novice programmers learning to code, less work has focused on how upper-division students use and trust these tools, especially while working with large code bases. In this study, we taught students about various GitHub Copilot features, including Copilot chat, in an upper-division software engineering course and asked students to add a feature to a large code base using Copilot. Our analysis revealed a novel interaction pattern that we call one-shot prompting, in which students ask Copilot to implement the entire feature at once and spend the next few prompts asking Copilot to debug the code or asking Copilot to regenerate its incorrect response. Finally, students reported significantly more trust in the code comprehension features than code generation features of Copilot, perhaps due to the presence of trust affordances in the Copilot chat that are absent in the code generation features. Our study takes the first steps in understanding how upper-division students use Github Copilot so that our instruction can adequately prepare students for a career in software engineering.

Providing timely and personalized feedback to large numbers of students is a long-standing challenge in programming courses. Relying on human teaching assistants (TAs) has been extensively studied, revealing a number of potential shortcomings. These include inequitable access for students with low confidence when needing support, as well as situations where TAs provide direct solutions without helping students to develop their own problem-solving skills. With the advent of powerful large language models (LLMs), digital teaching assistants configured for programming contexts have emerged as an appealing and scalable way to provide instant, equitable, round-the-clock support. Although digital TAs can provide a variety of help for programming tasks, from high-level problem solving advice to direct solution generation, the effectiveness of such tools depends on their ability to promote meaningful learning experiences. If students find the guardrails implemented in digital TAs too constraining, or if other expectations are not met, they may seek assistance in ways that do not help them learn. Thus, it is essential to identify the features that students believe make digital teaching assistants valuable. We deployed an LLM-powered digital assistant in an introductory programming course and collected student feedback ($n=813$) on the characteristics of the tool they perceived to be most important. Our results highlight that students value such tools for their ability to provide instant, engaging support, particularly during peak times such as before assessment deadlines. They also expressed a strong preference for features that enable them to retain autonomy in their learning journey, such as scaffolding that helps to guide them through problem-solving steps rather than simply being shown direct solutions.

Higher education is expanding in United States prisons, with a growing demand for STEM offerings. Academics from other disciplines have stressed the importance of culturally relevant pedagogy (CRP) in prison higher education, and computing in context has shown major benefits in CS1--- especially for women and nontraditional students. More work is needed to determine what contexts are relevant to incarcerated college students, and how to incorporate these into computing curricula. In this paper, we build on prior work on computing in context and culturally relevant techniques in computing. We analyze course data from a CS1 course taught in a college-in-prison program to answer the following research question: What contexts do incarcerated students in CS1 find relevant? We identify 24 topics pursued by students across 78 open-ended programming assignment submissions, the three most popular being business management, sports statistics, and physical health. These results offer insight into potential contexts that are meaningful to incarcerated college students to be incorporated into future computing curricula and interventions in prisons.

Generative AI (GenAI) has seen great advancements in the past two years and the conversation around adoption is increasing. Widely available GenAI tools are disrupting classroom practices as they can write and explain code with minimal student prompting. While most acknowledge that there is no way to stop students from using such tools, a consensus has yet to form on how students should use them if they choose to do so. At the same time, researchers have begun to introduce new pedagogical tools that integrate GenAI into computing curricula. These new tools offer students personalized help or attempt to teach prompting skills without undercutting code comprehension. This working group aims to detail the current landscape of education-focused GenAI tools and teaching approaches, present gaps where new tools or approaches could appear, identify good practice-examples, and provide a guide for instructors to utilize GenAI as they continue to adapt to this new era.

Live coding is a recommended teaching practice in which an instructor dynamically programs in front of students. However, findings related to students' engagement during live coding are mixed. Some works have reported that live codingseems to improve student engagement while others regard live coding as an activity in which students passively observe the instructor without asking questions or following along. Active live coding, in which students extend a live coding example and discuss with peers, incorporates active learning with the traditional live coding approach. We conducted a quasi-experimental study in which one section of an advanced introductory programming course was taught using active live coding (ALC) and the other was taught using traditional live coding (TLC). The goal of this work is to compare students' behavioral engagement in the two lectures using a classroom observation protocol called the Behavioral Engagement Related to Instruction (BERI) protocol. Our results from the 2,790 observations we collected indicate that traditional live coding engages only 65% of students, on average. However, we found a "persisting engagement'' effect of active live coding, where students were significantlymore engaged in the traditional live coding components of a lecture up to 20 minutesafter the active live coding component. Notably, the two lecture groups performed similarly on the Post-Lecture Questions, which were administered after each lecture as a review of the lecture material. Therefore, our results indicate an improved student engagement due to active live coding, but do not show a corresponding improvement in conceptual knowledge.

Since March of 2020, universities around the world have offered remote versions of courses to help limit the spread of COVID-19. Two years later, in the Spring 2022 quarter, the lectures in the CS1 course at our large, public research-intensive university were taught via two modalities---an in-person modality in which students attended traditional, in-person lectures and a blended modality in which students attended a remote lecture on Zoom. Every other course component---labs, discussions, office hours---were held in-person for both groups. The unique setup of the CS1 course allowed us to perform a comparative analysis of the outcomes and attitudes between the two groups. In this paper, we analyze the difference in course outcomes, peer support, competitive feelings in class, and students' sense of belonging between the groups. Our results indicate that students in the blended learning group attended lectures more frequently than their in-person counterparts yet performed 4-7%worse on the midterm and final exams. The blended learning group also experienced significantlyless feelings of competitiveness than their in-person counterparts. Interestingly, we discovered a consistent trend among our results indicating that the gap in grades, peer support, and classroom competitiveness between the blended group and in-person group was more pronounced among first- and second-year undergraduates than third- and four-year students. Despite the two learning groups having different instructors, our results shed light on the potential advantages and drawbacks of a blended learning experience in CS1 that instructors should consider when deciding on the format of their course.

The recent, widespread availability of Large Language Models (LLMs) like ChatGPT and GitHub Copilot may impact introductory programming courses (CS1) both in terms of what should be taught and how to teach it. Indeed, recent research has shown that LLMs are capable of solving the majority of the assignments and exams we previously used in CS1. In addition, professional software engineers are often using these tools, raising the question of whether we should be training our students in their use as well. This experience report describes a CS1 course at a large research-intensive university that fully embraces the use of LLMs from the beginning of the course. To incorporate the LLMs, the course was intentionally altered to reduce emphasis on syntax and writing code from scratch. Instead, the course now emphasizes skills needed to successfully produce software with an LLM. This includes explaining code, testing code, and decomposing large problems into small functions that are solvable by an LLM. In addition to frequent, formative assessments of these skills, students were given three large, open-ended projects in three separate domains (data science, image processing, and game design) that allowed them to showcase their creativity in topics of their choosing. In an end-of-term survey, students reported that they appreciated learning with the assistance of the LLM and that they interacted with the LLM in a variety of ways when writing code. We provide lessons learned for instructors who may wish to incorporate LLMs into their course.

Efforts to bring incarcerated and formerly incarcerated individuals into the field of computing stand to improve equitable access to both computing jobs, and consequently the benefits of our tools and innovations through the inclusion of more diverse perspectives. This report describes the design and execution of a college level introductory computing course conducted with 26 students currently incarcerated at a prison in the United States in Fall 2022. We discuss the ways that the prison environment and the student body differ from traditional college computing classes, and how this impacted the design and execution of the course. We found that despite significant environmental barriers to learning to program, such as not having access to a code interpreter, there were unique affordances of the student population, including maturity and community, that could be leveraged in the course design and policies. We conclude with many lessons learned for the purpose of improving future offerings of computing courses in prisons.

Over the past decades, computer science education (CSEd) research has studied the multitude of factors that may impact student success in introductory programming courses (CS0/CS1). The lack of foundational structure behind how these factors interrelate has made it difficult to gain a thorough understanding of this area of CSEd literature. Gaining a deeper understanding and applying structure to these factors would allow CSEd to adopt better teaching practices, study habits, learning environments, course materials, etc. and to better understand the student experience to better foster success among a broader population of students. Our systematic literature review used search criteria for factors that predicted student success in CS0/CS1, which yielded 311 research articles. We then mapped this body of work under the Biggs' 3P (Presage, Process, Product) educational model, which provides a comprehensive framework for how students engage with learning opportunities. We discovered that although many studies focused on the Presage and Product phases of the model, fewer studies mapped to the Process phase, which describes the students' active learning processes. Our study shows there is a potential gap in the literature and future studies should focus more specifically on how students choose to engage with learning opportunities and what factors may be hindering that engagement throughout a learning period.

Background and Context. Live coding is a teaching method in which an instructor dynamically writes code in front of students in an effort to impart skills such as incremental development and debugging. By contrast, traditional, static-code examples typically involve an instructor annotating or explaining components of pre-written code. Despite recommendations to use live coding and a wealth of qualitative analyses that identify perceived learning benefits of it, there are a lack of empirical evaluations to confirm those learning benefits, especially with respect to students’ programming processes.

Live coding---a pedagogical technique in which an instructor plans, writes, and executes code in front of a class---is generally considered a best practice when teaching programming. However, only a few studies have evaluated the effect of live coding on student learning in a controlled experiment and most of the literature relating to live coding identifies students' perceived benefits of live-coding examples. In order to empirically evaluate the impact of live coding, we designed a controlled experiment in a CS1 course taught in Python at a large public university. In the two remote lecture sections for the course, one was taught using live-coding examples and the other was taught using static-code examples. Throughout the term, we collected code snapshots from students' programming assignments, students' grades, and the questions that they asked during the remote lectures. We then applied a set of process-oriented programming metrics to students' programming data to compare students' adherence to effective programming processes in the two learning groups and categorized each question asked in lectures following an open-coding approach. Our results revealed a general lack of difference between the two groups across programming processes, grades, and lecture questions asked. However, our experiment uncovered minimal effects in favor of the live-coding group indicating improved programming processes but lower performance on assignments and grades. Our results suggest an overall insignificant impact of the style of presenting code examples, though we reflect on the threats to validity in our study that should be addressed in future work.

Recent research in computing has shown that student performance on prerequisite course content varies widely, even when students continue to progress further through the computing curriculum. Our work investigates instructors' perspectives on the purpose of prerequisite courses and whether that purpose is being fulfilled. In order to identify the range of instructor views, we interviewed twenty-one computer science instructors, at two institutions, that teach a variety of courses in their respective departments. We conducted a phenomenographic analysis on the interview transcripts, which revealed a wide variety of views on prerequisite courses. The responses shed light on various issues with prerequisite course knowledge, as well as issues around responsibility and conflicting pressures on instructors. These issues arise at the department level, as well as with individual course offerings.

Many institutions use undergraduate teaching assistants (tutors) in their computing courses to help provide more resources to students. Because of the role tutors play in students' learning experiences, recent work in computing education has begun to explore student-tutor interactions through the tutor's perspective and through direct observation of the interactions. The results suggest that these interactions are cognitively challenging for tutors and may not be as beneficial for students' learning as one might hope. Given that many of these interactions may be unproductive, this work seeks to understand how student expectations of these sessions might be impacting the interactions' effectiveness. We interviewed 15 students in a CS2 course to learn about the expectations and desires that students have when they attend tutoring sessions. Our findings indicate that there is variation in what students consider a desired result from the interaction, that assignment deadlines affect students' expectations and desires for interactions, and that students do not always want what they believe is beneficial for their learning. We discuss implications for instructors and potential guidance for students and tutors to make tutoring sessions more effective.

Incremental development is the process of writing a small snippet of code and testing it before moving on. For students in introductory programming courses, the value of incremental development is especially higher as they may suffer from more syntax errors, lack the proficiency to address complicated bugs, and may be more prone to frustration when struggling to correct code. However, to evaluate the effectiveness of interventions that aim to teach programming processes such as incremental development, we need to develop measures to assess such processes. In this paper, we present a way to measure incremental development. By qualitatively analyzing 15 student coding interviews, we identified common behaviors in the programming process that relate to incremental development. We then leveraged a dataset of over 1000 development sessions -- about 52,000 code snapshots at compilation time -- to automatically detect the common behaviors identified in our qualitative analysis. Finally, we crafted a formal metric, called the "Measure of Incremental Development'' (MID), to quantify how effectively a student used incremental development during a programming session. The MID detects common non-incremental development patterns such as excessive debugging after large additions of code to automatically assess a sequence of snapshots. The MID aligns with human evaluations of incrementality with over 80% accuracy. Our metric enables new research directions and interventions focused on improving students' development practices.

This session takes inspiration from the highly successful "Nifty Assignments" special session, but instead highlights high quality multiple-choice questions that can be used for Peer Instruction. Peer Instruction is a pedagogical practice characterized by asking students to answer challenging, conceptual questions in class. For each question, students individually respond, discuss the question in small groups, and respond again based on their new understanding. Peer Instruction has been widely identified as an important instructional technique in teaching computing. In this session, members of the community will present some of their best Peer Instruction questions along with a short explanation that provides the pedagogical content knowledge indicating why the question is a good question. If you are interested in learning more about Peer Instruction or finding new questions for your course(s), this session is for you.

As enrollments in computing courses have surged, the ratio of students to faculty has risen at many institutions. Along with many other large undergraduate programs, our institution has adapted to this challenge by hiring increasing numbers of undergraduate tutors to help students. In early computing courses, their role at our institution is primarily to help students with their programming assignments. Despite our institution offering a training course for tutors, we are concerned about the quality and nature of these student-tutor interactions. As instruction moved online due to COVID-19, this provided the unique opportunity to record all student-tutor interactions (among consenting participants) for research. In order to gain an understanding of the behaviors common in these interactions, we conducted an initial qualitative analysis using open coding followed by a quantitative analysis on those codes. Overall, we found that students are not generally receiving the instruction we might hope or expect from these sessions. Notably, tutors often simply give students the solution to the problem in their code without teaching them about the process of finding and correcting their own errors. These findings highlight the importance of tutoring sessions for learning in introductory courses and motivate remediation to make these sessions more productive.

Previous work in computing has shown that Black, Latinx, Native American and Pacific islander (BLNPI), women, first-generation, and transfer students tend to have worse outcomes during their time in university compared to their majority counterparts. Previous work has also found that students' incoming prerequisite course proficiency is positively correlated with their outcomes in a course. In this work, we investigate the role that prerequisite course proficiency has on outcomes between these groups of students.

A Concept Inventory (CI) is an assessment to measure student conceptual understanding of a particular topic. This article presents the results of a CI for basic data structures (BDSI) that has been previously shown to have strong evidence for validity. The goal of this work is to help researchers or instructors who administer the BDSI in their own courses to better understand their results. In support of this goal, we discuss our findings for each question of the CI using data gathered from 1,963 students across seven institutions.

Students’ sense of belonging has been found to be connected to student retention in higher education. In computing education, prior studies suggest that a hostile culture and a feeling of non-belonging can lead women, Black, Latinx, Native American, and Pacific Islander students to drop out of the computing field at a disproportionately high rate. Yet, we know relatively little about how computing students’ sense of belonging presents and evolves (if at all) through their college courses, particularly in courses beyond the introductory level, and little is known about how sense of belonging impacts student outcomes in computing. In an extension of a previous study, we examined students’ sense of belonging in six early undergraduate computer science courses across three consecutive quarters at a large research-intensive institution in North America. We found that women and first generation students have a lower incoming sense of belonging across all courses. When exploring sense of belonging’s tie to student outcomes we found that lower sense of belonging was correlated with negative course outcomes in terms of pass rates and course performance. We also found that it is less tied to student performance as students get further into the CS curriculum. Surprisingly, there was no indication that sense of belonging is predictive of retention in terms of persistence to the next CS course outside of the first course in our two-course CS1 sequence.

Computer science students struggle in early computing courses as evinced by high failure rates and poor retention. As such, studies have attempted to characterize the root of student struggles from many perspectives, including cognitive, meta-cognitive, and social emotional. Typically, studies have limited their inquiry to a specific perspective or a single course. This paper reports the results of a broad student experience survey conducted across several computer science courses. Through a periodic survey, students rated various cognitive, socio-emotional, external, personal, and structural barriers in terms of how much each impacted their learning throughout the term. An exploratory factor analysis of these questions revealed four factors—personal obligations, lack of sense of belonging, in-class confusion, and lack of confidence—that capture a range of possible struggles students may face. We analyzed the prevalence of these factors across courses, performance quartiles, and demographic groups broken down by gender, race/ethnicity, and matriculation status. Students in lower performance quartiles report higher stress levels on multiple factors, with statistically significant differences found between all quartiles and courses, for most factors. Moreover, students from traditionally underrepresented groups report struggling more across all four factors, suggesting that they may be facing more challenges than classmates from represented populations. Overall, these findings indicate that student struggles are associated with stresses from many areas of their lives, suggesting that future interventions should target multiple areas of stress.

Although Massive Open Online Courses have the potential to reach a much broader audience and offer a lower cost education than traditional in-person classes, they have struggled with low completion rates and low diversity amongst those enrolled and completing the courses. In 2015, we built a series of online courses in computing with the specific goal of attracting and retaining students from groups underrepresented in computing. In our design, we incorporated a number of features aimed at improving the inclusive nature of the courses including: a project-centered course design; an online version of Peer Instruction ConceptTests; videos where students, faculty, and professionals report their struggles when they first learned computing concepts; videos by professional software engineers explaining how computing concepts from the course are used in industry; and videos aimed at providing additional support on the project to students who might be struggling. In this work, we report on the design of the courses and examine how successful our courses were at attracting and retaining women students. We find that compared to other computing courses offered by our institution on the same platform, our courses have: a higher percentage of women enrollment, higher rates of course completion for both men and women, and a slightly smaller gap between completion rates for men and women.

Spring 2020 brought enormous change to student learning, as universities scrambled to put into place support structures to aid students' learning in a remote context. Computer science education was both well-positioned for this change and faced unique challenges, e.g. that students often need significant (in-person) support with programming. In this study we examine how aspects of students' remote learning experience in spring 2020 compared to the same aspects in previous in-person, pre-COVID offerings of 6 lower- to mid-division computer science courses at UC San Diego (UCSD), a large US research university. We were in a unique position to make this comparison because we had been collecting data on several aspects of students' course experiences throughout the 2019-2020 academic year. We found, surprisingly, that most elements of students' experiences that we examined were unchanged, or even improved, in spring 2020. Students in spring reported similar or lower stress levels and found their courses similarly or less challenging relative to previous quarters. However, some aspects did degrade. Students had less connection with their peers (particularly in introductory classes), more interference from family obligations, and higher drop/fail rates in some classes. Surprisingly, these results hold across all assessed demographics. Our results indicate that the actions UCSD and its CS instructors took to mitigate the stresses of remote learning in spring 2020 were largely successful and provide implications for improving education beyond the pandemic.

Our prior work found differences in study habits between high- and low-performers in a small-scale qualitative study, and this work seeks to verify and extend these findings by examining the study habits of a larger population of CS1 students. To do this, we devised a survey based on the findings of our prior qualitative study. The responses of CS1 students reveals that some study habits are more frequently practiced by higher-performers then lower-performers or vice versa. One concern with these findings is that the differences in study habits might simply be explained by prior experience. As such, we compare study habits between students with and without prior experience as well. We find that although prior experience translates to better class performance, it is not associated with the same study habits as lower- and higher-performers, suggesting that prior experience and study habits are separately associated with better student performance. These findings encourage further inquiry into the role of study habits in student success and whether explicit instruction on better study habits might be the basis for successful future interventions.

One of the goals of computing education research is to document the potential strengths and weaknesses of contemporary teaching methods in computing. Live coding has recently gained attention as one of the best practices for teaching programming. To offer a more comprehensive understanding of the existing body of research about live coding, we reviewed papers in computing education research that investigated the value of live coding in an educational setting. We categorized each paper based on (1) how it defines live coding, (2) whether its version of live coding could be considered active learning, (3) the type of study conducted, (4) types of data collected and the data analysis methods used, (5) evidence provided for the effectiveness of live coding, (6) reported benefits and drawbacks of live coding, and (7) reported theoretical frameworks used to explain the basis, effects or goals of live coding. We found that although live coding has been recommended as one of the best practices for teaching programming, there is a lack of empirical evidence to support claims about the effectiveness of live coding on student learning. Finally, we discuss the implications of our findings and suggest future research directions that could develop a more holistic understanding of this pedagogical technique.

Previous studies show that CS students may not learn as much from their courses as we might expect. This could have ramifications on how students succeed in their future careers and may explain why researchers report a gap between industry expectations and the abilities of recent CS graduates. However, previous studies have also shown that students improve their prerequisite knowledge in subsequent courses. This study investigates the introductory data structures proficiency of students in different courses at various stages in our CS program, employing the validated Basic Data Structures Inventory (BDSI). Additionally, we investigate whether subpopulations, including transfer students and underrepresented groups, may be more prone to not attaining as much knowledge from our courses as we might expect. We find that students' knowledge of basic data structures is, on average, better in later courses. However, we also find subpopulations of students that perform worse than others or seem to not improve their knowledge in later courses. Specifically, we find students that transferred to our institution from a different school perform significantly worse on the BDSI than other students and do not improve their BDSI performance in later courses. We also find students from demographic backgrounds that are underrepresented in computing scored slightly, though not statistically significantly, worse than others. Our findings warrant future investigations on how our programs can better serve the students in the affected subpopulations.

Computer Science now has a number of validated instruments available for measuring student knowledge or interest in computing including the Second CS1 Assessment (SCS1), The Basic Data Structures Inventory (BDSI), the Computing Attitudes Survey (CAS), and the Digital Logic CI. These instruments can be used by instructors to assess their students and/or their own teaching. They can also be used by researchers to measure students' learning or attitudes. The goal of this BOF is to help instructors and researchers gain a better understanding of how to use these instruments, whether that be to get started in education research, to compare student learning across terms/curricular revisions, or just to learn more about student misconceptions. We will begin by discussing the available instruments, their purpose, and how to obtain them. Then we will open the discussion to the group on what they would like to measure, how these instruments might work for them, and how to best employ them with their students.

This panel explores experiences and insights from three successful propagators into how others can successfully encourage more wide-spread use of their innovations. Issues covered include designing for dissemination, techniques for recruiting potential adopters, suggestions for convincing faculty to try an innovation and continue using it, and identifying points of friction and overcoming resistance from administrators, students, and/or peers. These topics are discussed and illustrated with personal experiences and anecdotes from our illustrious panelists.

Over a decade ago, the CS1 course for students without prior programming experience at a large research-intensive university was redesigned to incorporate three best practices in teaching programming: Media Computation, Pair Programming, and Peer Instruction. The purpose of this revision was to improve the quality of the course, appeal to a larger student body, and improve retention in the major. An initial analysis of the course indicated an increase in pass rates and 1-yr retention of students in the major. Now that time has passed and those students impacted by the revision have had time to graduate, this longitudinal study revisits and expands on these prior findings through examining student outcomes over a twelve year period (2001 through 2013). The student outcomes examined include failure rates in CS1, retention rates in the major, rates of switching into the major, time to degree, and performance in subsequent major courses. We compare these findings against similar metrics collected for another CS1 course at the same institution that caters to students with prior programming experience and did not make changes during this same time period. Overall, the inclusion of media computation, pair programming, and peer instruction corresponds to a significant improvement in passing rates for CS1 as well as retention of majors from CS1 through graduation. In turn, there is no indication that this larger group of students experienced any harm in terms of lower grades in upper-division courses or their time to degree.

Although elements of the academia-industry gap have been studied extensively, these studies have mostly ignored the primary stakeholder for changing academia: faculty. Building on a recent qualitative study that revealed a wide range of faculty views on the gap, this study quantitatively examines faculty views through a survey on the goals of CS education, how CS programs should address the academia-industry gap, and which barriers prevent adoption of remedies. Analysis of the 249 responses reveals that a majority of faculty share common goals in supporting student preparation for a career in industry. Moreover, faculty strongly view their own institutions as the prime party responsible for student preparation for careers in both academia and industry. We also find that whereas faculty are generally in agreement on what could be improved to provide students with better industry preparation, some reported far greater barriers to implementing those improvements than others.

In computer science classes it can be a challenge to ensure every student has a functioning development environment. Running pre-configured servers that provide students with remote access can help mitigate most of these setup issues, however they can also introduce new limitations of their own. We propose using DevContainers to overcome the local machine setup difficulties for the students. DevContainers allow the instructional staff to provision a development environment (a Docker image) with all the correct software versions pre-configured. This development environment can be used on any major OS through Docker. Moreover, through this DevContainer configuration, Microsoft Visual Studio Code can integrate seamlessly with the Docker container to provide an experience for the user that is practically the same as working on the native OS. This work examines the value of employing a DevContainer setup in an Advanced Data Structures course and provides details for those interested in using DevContainers in their courses.

Computer Science now has a number of validated instruments available for measuring student knowledge or interest in computing (SCS1, BDSI, CAS, Digital Logic CI, etc.). But when and how should instructors and researchers use these instruments? In this BOF, we will begin by discussing the available instruments, their purpose, and how to obtain them. Then we will open the discussion to the group on what they would like to measure, how these instruments might work for them, and how to best employ them with their students.

Given the long-standing concern about students failing introductory programming courses, there is a need for interventions that may aid those students. In this work, we examine the potential benefit of three interventions based on prior computing education research (CER) or STEM education research literature: mindset interventions, the use of "Thinkathons" as an alternative to programming labs, and metacognitive interventions to encourage more productive study habits. We conducted an in-class study that controlled for both time-on-task and selection bias to investigate the potential benefits of integrating these interventions into the existing footprint of an introductory computing course. Despite the previously reported promise of the interventions we implemented, our findings were that in this context these techniques had only a mild positive effect for some students. We discuss possible reasons why these techniques are less successful than instructors might hope and argue for the need for more research on this topic.

Recent work has identified a mismatch between instructor expectations of students' mastery of prerequisite course content and their actual ability. This invites the question of why this mismatch exists. We first examined grades in prerequisite courses and found they meaningfully correlated with performance on an assessment testing their knowledge of prerequisite material. In addition, we found neither taking alternatives to the primary identified prerequisites nor the delay between taking prerequisite courses and the follow-on course meaningfully impacts performance. Second, we confirmed that prerequisite course grades are significantly correlated with the grade in the follow-on course-confirming that the grades in the previous courses convey some information about student understanding of those topics. Perhaps surprisingly, we found that grades in courses outside computing were similarly correlated as those courses inside computing, suggesting that underlying factors such as general study skills may be as important as the domain-specific knowledge itself.

The 32 chapters of the 2019 Cambridge Handbook of Computing Education Research synthesize the existing research in computing education and propose new directions for future research. An author from each chapter will summarize their chapter with auto-advancing slides. Attendees will be introduced to the breadth of content in the new handbook and can identify chapters of interest. This fits uniquely as a special session, and will likely be informative, inspiring, and overwhelming.

The Impostor Phenomenon (IP) is often discussed as a problem in the field of computer science, but there has yet to be an empirical study to establish its prevalence among CS students. One survey by the Blind app found that a high number of software engineers at some of the largest technology companies self-reported feelings of Impostor Syndrome; however, self-reporting of Impostor Syndrome is not the standard diagnostic for identifying whether an individual exhibits feelings of the Impostor Phenomenon. In this work, the established Clance IP Scale is used to identify the prevalence of IP among graduate and undergraduate computer science students at a large research-intensive North American institution. Among this population of over 200 students, 57% were found to exhibit frequent feelings of the Impostor Phenomenon with a larger fraction of women (71%) experiencing frequent feelings of the Imposter Phenomenon than men (52%). Additionally, IP was found to have greater prevalence among computer science students than among students of other populations from comparable studies. Due to the negative impacts associated with feelings of the Impostor Phenomenon, computer science education should work to improve student awareness and help student cope with these feelings.

Concept inventories (CIs) allow researchers and practitioners to measure student conceptual learning within a course or topic area. While they have enabled meaningful pedagogical change in other disciplines, there are relatively few CIs in computer science. In this paper, we report on our experiences as recent developers of a CI for basic data structures. We discuss each step along the route to a CI and offer tips based on what we have learned. We encourage others to create CIs, and we hope that this paper will serve as a practical guide through the process.

Previous work has found that recent computer science graduates often experience difficulty transitioning into their new roles in industry due to a significant gap between their academic experiences and industry's expectations. Although multiple studies have identified the views of students and members of industry on the value of a CS degree as preparation for industry, the faculty perspective on this topic remains unclear. Understanding these views could shed light on why the academia-industry gap has persisted despite the attention. This study identified faculty views on the goals of an undergraduate education and a CS major, focusing on preparation for careers in industry. In order to identify a spectrum of faculty views, we interviewed 14 faculty from a variety of backgrounds across three institutions. A phenomenographic analysis of the transcripts reveals that many faculty believe that industry preparation is an important programmatic goal, yet they encounter significant resource obstacles to achieving that goal.

A Concept Inventory (CI) is a validated assessment to measure student conceptual understanding of a particular topic. This work presents a CI for Basic Data Structures (BDSI) and the process by which the CI was designed and validated. We discuss: 1) the collection of faculty opinions from diverse institutions on what belongs on the instrument, 2) a series of interviews with students to identify their conceptions and misconceptions of the content, 3) an iterative design process of developing draft questions, conducting interviews with students to ensure the questions on the instrument are interpreted properly, and collecting faculty feedback on the questions themselves, and 4) a statistical evaluation of final versions of the instrument to ensure its internal validity. We also provide initial results from pilot runs of the CI.

As computer science enrollments continue to surge, exam grading requires significant instructional resources. Online grading platforms have been developed in recent years and have been adopted at a number of institutions; however, their effectiveness compared with traditional grading on paper is not fully known. This study is the first in CS to compare online and paper grading. Comparing overall time to grade, including all factors, online grading doesn't show a consistent advantage compared with paper grading. We observed that online grading is much faster during the actual grading phase, but some of this benefit is offset by the additional overhead prior to grading (e.g., scanning) for online grading. Examining student and grader preferences based on feedback, both groups show a strong preference for the online format, predominately due to the convenience of the online platform. Graders report being able to grade more accurately online with the ability to modify rubrics, but that grading on paper tends to result in more social interactions among graders.

Although recent work in computing has discovered multiple techniques to identify low-performing students in a course, it is unclear what factors contribute to those students' difficulties. If we were able to better understand the characteristics of such students, we may be better able to help those students. This work examines the characteristics of low- and high-performing students through interviews with students from an introductory computing class. We identify a number of relevant areas of student behavior including how they approach their exam studies, how they approach completing programming assignments, whether they sought help after identifying misunderstandings, how and from whom they sought help, and how they reflected on assignments after submitting them. Particular behaviors within each area are coded and differences between groups of students are identified.

As we celebrate the 50th SIGCSE Symposium, this panel explores how computing education researchers chart a course individually and as a community to build our research practices and collective knowledge of computing education. This navigation involves developing our research goals, which tools we use to work towards those goals, and which academic communities outside of computing education we seek to learn from and contribute to. However, these processes of navigation are rarely discussed as a community. Paper and grant submissions and reviews provide an imperfect way for our community to communicate our varied values and priorities. This panel brings together experts in computing education research who differ in their research goals, tools, and external communities. We can expect a lively discussion amongst the panelist and we hope to spark important discussions within the computing education research community!

A number of recent studies in computer science education have explored the value of various data sources for early prediction of students' overall course performance. These data sources include responses to clicker questions, prerequisite knowledge, instrumented student IDEs, quizzes, and assignments. However, these data sources are often examined in isolation or in a single course. Which data sources are most valuable, and does course context matter? To answer these questions, this study collected student grades on prerequisite courses, Peer Instruction clicker responses, online quizzes, and assignments, from five courses (over 1000 students) across the CS curriculum at two institutions. A trend emerges suggesting that for upper-division courses, prerequisite grades are most predictive; for introductory programming courses, where no prerequisite grades were available, clicker responses were the most predictive. In concert, prerequisites and clicker responses generally provide highly accurate predictions early in the term, with assignments and online quizzes sometimes providing incremental improvements. Implications of these results for both researchers and practitioners are discussed.

While it is widely believed that taking a class's prerequisites is critical for success, less is known about how proficiency with the prerequisite knowledge from those courses affects performance in later courses. Specifically, it is unclear how well students understand material from prerequisite courses and whether that understanding may impact their outcomes in the subsequent course. Additionally, in subsequent courses, do students strengthen their knowledge from prerequisite courses and, if they do, does that improvement matter for the subsequent course? This study examines the prerequisite knowledge of 208 students in an upper-division data structures class at a large North American research university. Prerequisite proficiency on entry to the course was surprisingly low, with nearly a third of students demonstrating low proficiency and only a quarter high proficiency. Students modestly improved their proficiency during the term, lifting a third of those with low proficiency to at least medium proficiency. Overall, final exam performance was significantly correlated with prerequisite knowledge. For those with low initial proficiency, improvement in proficiency was significantly correlated with performance on the final. These results suggest that more attention needs to be placed on reinforcing prerequisite knowledge for those with low proficiency.

As enrollments and class sizes in postsecondary institutions have increased, instructors have sought automated and lightweight means to identify students who are at risk of performing poorly in a course. This identification must be performed early enough in the term to allow instructors to assist those students before they fall irreparably behind. This study describes a modeling methodology that predicts student final exam scores in the third week of the term by using the clicker data that is automatically collected for instructors when they employ the Peer Instruction pedagogy. The modeling technique uses a support vector machine binary classifier, trained on one term of a course, to predict outcomes in the subsequent term. We applied this modeling technique to five different courses across the computer science curriculum, taught by three different instructors at two different institutions. Our modeling approach includes a set of strengths not seen wholesale in prior work, while maintaining competitive levels of accuracy with that work. These strengths include using a lightweight source of student data, affording early detection of struggling students, and predicting outcomes across terms in a natural setting (different final exams, minor changes to course content), across multiple courses in a curriculum, and across multiple institutions.

To be effective instructors and CS education researchers, we must identify and understand student difficulties surrounding core computing topics. This study examines student difficulties with the basic data structures commonly found in CS2 courses. Initial exploration of student thinking began with think-aloud interviews with students. These interviews centered on open-ended questions that were iteratively improved upon based on analysis of interview transcripts. The revised open-ended questions were then posed to 249 students during an end-of-term final exam study session. Using the explanations and justifications included by students, responses to the questions were coded and summarized. This work characterizes the difficulties revealed by student responses, and provides details of their prevalence among the examined student population.

Jigsaw learning is a cooperative learning technique enabling students to teach and learn from their peers. Although prior studies investigated the efficacy of Jigsaw learning in computing education by measuring student performance after Jigsaw activities, this work reports on student and instructor experiences with Jigsaw learning. Jigsaw activities were performed in lectures throughout the term and student experience data was collected through student surveys. The survey results reveal that 72% of survey respondents thought the Jigsaw activities helped their learning. Furthermore, 75% of respondents acknowledged their role in class as a more engaged learner and 44% of those students identified themselves as taking on the role of a teacher. The instructor found Jigsaw learning to be labor intensive but observed higher levels of student engagement.

Peer Instruction (PI) is an active learning pedagogy that has been shown to improve student outcomes in computing, including lower failure rates, higher exam scores, and better retention in the CS major. PI's key classroom mechanism is the PI question: a formative multiple choice question on which students vote, then discuss, then vote again. While research indicates that PI questions lead to learning gains for students, relatively little is known about the questions themselves and how faculty employ them. Additionally, much of the work has examined PI data collected by researchers operating in a quasi-experimental setting. We examine data collected incidentally by multiple instructors using PI as a pedagogical technique in their classroom. We look at how many questions instructors use in their courses, the difficulty level of the questions, and normalized gain, a metric that looks at increases in student correctness between individual and group votes. We find normalized gain levels similar to those in existing literature, indicating that students are learning, and that most questions, even those developed by instructors new to PI, fall within recommended difficulty levels, indicating instructors can create good PI questions with little training. We also find that instructors add PI questions over the first several iterations of a new PI course, showing that they find PI questions valuable and suggesting that full development of PI materials for a course may take multiple semesters.

Peer Instruction is an active learning pedagogy that has been shown to improve student outcomes in computing, including lower failure rates, higher exam scores, and better retention in the CS major. A key classroom mechanism for Peer Instruction is the "clicker question": a formative multiple-choice question on which students vote, then discuss, then vote again. While research indicates that clicker questions lead to learning gains for students, relatively little is known about the questions themselves and how faculty employ them. Additionally, much of the work has examined clicker data collected by CS Education researchers operating in a quasi-experimental setting. In this project, we examine clicker data collected incidentally by multiple instructors using clickers as a pedagogical technique in their classroom. This work represents a first effort to systematically evaluate how instructors use clicker questions, including how many clicker questions are used in a course, how difficult the questions used are, and whether instructors add or modify questions over time.

With surging enrollments in computer science, large classes are becoming standard, even at the upper division. Unfortunately, this new reality can leave students feeling anonymous and unsupported. This work examines the impact of several lightweight interventions on students' sense of connection with instructors and the class. These strategies were employed in a range of large courses at a public research-focused university. The implemented techniques include: opportunities for one-on-one tutoring, seating assignments with consistent teaching staff members seated in class, and assigned small discussion sections, among others. All strategies are lightweight and require only the usual staffing resources afforded to a class. In this report, we evaluate student sense of community and reflect on the benefits and challenges of these techniques.

Establishing learning goals for a course allows instructors to design course content to address those goals, helps students to focus their learning appropriately, and enables researchers to assess learning of those goals. In this work, we propose six learning goals for a topic prevalent in CS2 courses: Basic Data Structures. These learning goals arise from reviewing several CS2 courses at a variety of institutions, surveying faculty experts who commonly teach CS2, and meeting and working closely with these experts. We outline our process for creating learning goals, identify important topics underlying these goals, and provide examples of how the goals developed on the path to consensus. We also document that the term "CS2" does not have a unified interpretation within the CS education community and describe how this hurdle influenced our decision to focus on Basic Data Structures.

Replication research is rare in CS education. For this reason, it is often unclear to what extent our findings generalize beyond the context of their generation. The present paper is a replication and extension of Achievement Goal Theory research on CS1 students. Achievement goals are cognitive representations of desired competence (e.g., topic mastery, outperforming peers) in achievement settings, and can predict outcomes such as grades and interest. We study achievement goals and their effects on CS1 students at six institutions in four countries. Broad patterns are maintained --- mastery goals are beneficial while appearance goals are not --- but our data additionally admits fine-grained analyses that nuance these findings. In particular, students' motivations for goal pursuit can clarify relationships between performance goals and outcomes.

Collaborative exams have shown promise for improving student learning in computing. Prior studies have focused on benefits for all students, whereas this study seeks to refine our understanding of which students benefit and how group composition impacts that benefit. Using a crossover experimental design, the study first investigates whether students from differing performance levels (low, medium, or high) benefit from the collaborative exam. We find that students in the middle of the class (neither high nor low performers) tend to benefit strongly from the collaborative exam. Second, we explore whether group composition based on performance levels impacts the performance of members of the group. The results suggest more homogeneous groups (i.e., students in the group are at similar performance levels) are beneficial whereas students in groups with high heterogeneity do not experience significant performance differences between the pre-test and post-test.

As class-sizes grow in computer science, the personal attention received by students tends to diminish. This work aims to replicate small-class community effects within a large class by creating "micro-classes"---small groups within the large class. These micro-classes consist of 20--30 students led by graduate teaching assistants and undergraduate tutors who are specifically trained in small-classroom instructional techniques. This paper studies the outcomes of the micro-classes framework in an upper-division data structures course and compares them to outcomes from the same class taught in a large lecture, active-learning format. Students report increased satisfaction and a higher perception of community in the micro-classes section, though there was no discernible difference in student academic performance.

Collaborative group exams, including two-stage exams, have received increased attention in other disciplines after studies have shown their value for student learning. In computer science, prior work has shown students may value two-stage exams, but their impact on student learning in computing is unknown. In this randomized, crossover study, student learning on four topics during two-stage midterm exams is examined. Specifically, students are randomly selected to answer questions on a topic either working in groups or individually during the exam. On a quiz two weeks later, those who worked in groups statistically significantly outperform those who worked individually. The metric of performance comparison is normalized learning gain. These are short term gains, however, as the benefit is no longer apparent by the final exam. This is the first study to use controlled experiments to examine the value of the group stage of two-stage exams in computing.

As student enrollments in computer science increase, there is a growing need for pedagogies that scale. Recent evidence has shown Peer Instruction (PI) to be an effective in-class pedagogy that reports high student satisfaction even with large classes. Yet, the question of the scalability of traditional lecture versus PI is largely unexplored. To explore this question, this work examines publicly available student evaluations of computer science courses across a wide range of class sizes (50--374 students) over a four year period. It first compares evaluations regardless of size and confirms prior work that PI classes are better appreciated by students than traditional lecture. It then examines how course evaluations change with class size and provides evidence that PI achieves a smaller decline in evaluations as class size increases.

In this workshop participants will receive an overview of teaching practices in computer science that research indicates are effective. While the field of computer science education is young, it has uncovered several teaching practices that can be adopted by instructors that can improve both the retention and performance of students. These evidence based teaching practices include active learning techniques such as peer instruction and prior-knowledge activities, pair programming, and use of subgoal labels. Participants will experience firsthand many of these techniques and will be provided with resources on where to find more information, including the original research papers, on each technique. If you want to attend a workshop that will have an immediate impact in your class -- attend this one. The workshop will be interactive, engaging, and show you how to incorporate teaching practices that are empirically proven to provide benefits. You are guaranteed to leave with a list of many freely available resources and ideas to use in your next class. You will also have the opportunity to "ask the experts" as the authors of many of these research papers will be leading that session of the workshop.

With growing enrolment and ongoing research into best practices for team work, many instructors are rethinking how to form, evaluate, and manage teams. In this BoF, instructors will discuss the strategies they have employed, the effectiveness of those approaches, and the tools that support administering teams. Discussion topics may include self- and peer-evaluation, grading strategies, software support, and conflict resolution.

Although computing students may enjoy when their instructors teach using analogies, it is unknown to what extent these analogies are useful for their learning. This study examines the value of analogies when used to introduce three introductory computing topics. The value of these analogies may be evident during the teaching process itself (short term), in subsequent exams (long term), or in students' ability to apply their understanding to related non-technical areas (transfer). Comparing results between an experimental group (analogy) and control group (no analogy), we find potential value for analogies in short term learning. However, no solid evidence was found to support analogies as valuable for students in the long term or for knowledge transfer. Specific demographic groups were examined and promising preliminary findings are presented.

Being able to identify low-performing students early in the term may help instructors intervene or differently allocate course resources. Prior work in CS1 has demonstrated that clicker correctness in Peer Instruction courses correlates with exam outcomes and, separately, that machine learning models can be built based on early-term programming assessments. This work aims to combine the best elements of each of these approaches. We offer a methodology for creating models, based on in-class clicker questions, to predict cross-term student performance. In as early as week 3 in a 12-week CS1 course, this model is capable of correctly predicting students as being in danger of failing, or not, for 70% of the students, with only 17% of students misclassified as not at-risk when at-risk. Additional measures to ensure more broad applicability of the methodology, along with possible limitations, are explored.

The programming education literature includes many observations that pass rates are low in introductory programming courses, but few or no comparisons of student performance across courses. This paper addresses that shortcoming. Having included a small set of identical questions in the final examinations of a number of introductory programming courses, we illustrate the use of these questions to examine the relative performance of the students both across multiple institutions and within some institutions. We also use the questions to quantify the size and overall difficulty of each exam. We find substantial differences across the courses, and venture some possible explanations of the differences. We conclude by explaining the potential benefits to instructors of using the same questions in their own exams.

Ten selected questions have been included in 13 introductory programming exams at seven institutions in five countries. The students' results on these questions, and on the exams as a whole, lead to the development of a benchmark against which the exams in other introductory programming courses can be assessed. We illustrate some potential benefits of comparing exam performance against this benchmark, and show other uses to which it can be put, for example to assess the size and the overall difficulty of an exam. We invite others to apply the benchmark to their own courses and to share the results with us.

Peer Instruction (PI) is a student-centric pedagogy in which students move from the role of passive listeners to active participants in the classroom. Over the past five years, there have been a number of research articles regarding the value of PI in computer science. The present work adds to this body of knowledge by examining outcomes from seven introductory programming instructors: three novices to PI and four with a range of PI experience. Through common measurements of student perceptions, we provide evidence that introductory computing instructors can successfully implement PI in their classrooms. We find encouraging minimum (74%) and average (92%) levels of success as measured through student valuation of PI for their learning. This work also documents and hypothesizes reasons for comparatively poor survey results in one course, highlighting the importance of the choice of grading policy (participation vs. correctness) for new PI adopters.

No abstract available.

Achievement goals are cognitively-represented end states that individuals strive to reach in competence situations. Well-studied by educational psychologists, achievement goals are robust predictors of grades, interest, and motivation of students. In this paper, we apply achievement goal theory to measure CS1 students' achievement goals and consequent interest in CS and final exam grade. We find that students aiming for topic mastery become interested in CS and, contrary to theoretical expectations, perform well on the exam. A more complex pattern of results surrounds students who orient toward competence demonstration or normative comparison, and the link between such performance goals and outcomes is less clear. We argue for the continued appropriation of educational theory to inform our studies of CS success.

Recent work in computing has converged on a collection of complementary findings suggesting the value of the Peer Instruction (PI) pedagogy. Compared to lecture, PI has been shown to decrease fail rates, increase final exam grades, and increase engagement and enjoyment. In PI, students work together to exchange perspectives and use clickers to answer challenging conceptual questions in the presence of a knowledgeable instructor.

Summary The current state of practice in supercomputer resource allocation places jobs from different users on disjoint nodes both in terms of time and space. While this approach largely guarantees that jobs from different users do not degrade one another's performance, it does so at high cost to system throughput and energy efficiency. This focused study presents job striping, a technique that significantly increases performance over the current allocation mechanism by colocating pairs of jobs from different users on a shared set of nodes. To evaluate the potential of job striping in large‐scale environments, the experiments are run at the scale of 128 nodes on the state‐of‐the‐art Gordon supercomputer. Across all pairings of 1024 process network‐attached storage parallel benchmarks, job striping increases mean throughput by 26% and mean energy efficiency by 22%. On pairings of the real applications Gyrokinetic Toroidal Code (GTC), Large‐scale Atomic/Molecular Massively Parallel Simulator (LAMMPS), and MIMD Lattice Computation (MILC) at equal scale, job striping improves average throughput by 12% and mean energy efficiency by 11%. In addition, the study provides a simple set of heuristics for avoiding low performing application pairs. Copyright © 2013 John Wiley & Sons, Ltd.

This session is a chance for researchers studying concept inventories (CIs)--low-cost assessments highlighting student misconceptions in a field--and CS education practitioners to communicate about advances in concept inventories in an engaging and utterly ridiculous way.

Recent computer science education research using isomorphic questions in Peer Instruction (PI) classes demonstrates that students learn from talking to their peers and listening to the instructor's follow-up explanation. These results provide evidence of the effectiveness of the PI process but are limited to what happens in a single class session. The present work extends the use of isomorphic questions to investigate how in-class learning translates to success on the final exam. Despite considerable time between in-class questions and the final exam, we find that students who learn in class are shown to retain that learning and to perform better than students who did not learn in class. In addition, compared to students already understanding the material, those who learned the material in class are almost (87%) as likely to correctly answer isomorphic exam questions. Our results have implications for the value of difficult PI questions and the meaning of in-class response graphs.

Recent work in computing has converged on a collection of complementary findings suggesting the value of the Peer Instruction (PI) pedagogy. Compared to lecture, PI has been shown to decrease fail rates, increase final exam grades, and increase engagement and enjoyment. In PI, students work together to exchange perspectives and use clickers to answer challenging conceptual questions in the presence of a knowledgeable instructor.

Recent research suggests that the first weeks of a CS1 course have a strong influence on end-of-course student performance. The present work aims to refine the understanding of this phenomenon by using in-class clicker questions as a source of student performance. Clicker questions generate per-lecture and per-question data with which to assess student understanding. This work demonstrates that clicker question performance early in the term predicts student outcomes at the end of the term. The predictive nature of these questions applies to code-writing questions, multiple choice questions, and the final exam as a whole. The most predictive clicker questions are identified and the relationships between these questions and final exam performance are examined.

Concept Inventory (CI) assessments, which target high-level learning goals, have proven highly valuable for higher education research. These assessments have helped to evaluate pedagogical practices among individual instructors, both within and across institutions, and have hence elevated the level of discourse on education within the community. The success of CIs in physics has inspired similar developments in computer science, with a few CIs now developed for computer science courses. However, the development of a CI typically follows a burdensome process, requiring a significant investment to produce a single CI that may be difficult to deploy due to institutional curricular differences. Furthermore, as our field continues to be shaped by technological advances, a path to faster, more modular CI development is critical.

In computer science, the active learning pedagogical practice of Peer Instruction (PI) has been shown to improve final exam performance, reduce student failure rates, and improve student retention. PI consists of two major parts: group discussion and follow-up instructor intervention. We expect that PI performance as a whole will correlate with final exam performance, but it is unclear whether or how each piece of PI is involved in these relationships. In this work, we use isomorphic questions to isolate the effects of peer discussion and instructor intervention, and examine scores on a final exam and its code-writing and code-tracing questions. We find that both pieces of PI correlate with the final exam as a whole, code-tracing question (similar to PI questions), and code-writing question (not similar to PI questions). This is further evidence that both PI components are important to the success of PI.

There is a growing body of evidence showing active learning pedagogies are effective for ensuring long-term student learning, reducing course failure rates, and retaining majors. While active learning pedagogies are more often employed in lower divisional CS courses, systems classes-such as OS, architecture, and networks-are just beginning to see a shift from standard lecture to active learning. This BOF will provide a forum for sharing ideas of how to integrate active learning techniques into systems courses, using the following questions as starting points:

It is generally assumed that early success in CS1 is crucial for success on the exam and course as a whole. Particularities of students, densely-connected CS1 content, and recurring core topics each suggest that it is difficult to rebound from early misunderstandings. In this paper, we use Peer Instruction (PI) data, in addition to exam data, to explore relationships between in-class assessments and performance at the end of term and on the exam. We find that early course performance very quickly and strongly predicts performance on the final exam and that subsequent weeks provide no major increase in that predictive power. In contrast, early performance is similarly predictive of performance in the last weeks of PI questions, but subsequent weeks are increasingly more predictive. We speculate on what this means for the content of these assessments and potential future assessment practices.

This work presents an end-to-end methodology for quantifying the performance and power benefits of simultaneous multithreading (SMT) for HPC centers and applies this methodology to a production system and workload. Ultimately, SMT’s value system-wide depends on whether users effectively employ SMT at the application level. However, predicting SMT’s benefit for HPC applications is challenging; by doubling the number of threads, the application’s characteristics may change. This work proposes statistical modeling techniques to predict the speedup SMT confers to HPC applications. This approach, accurate to within 8%, uses only lightweight, transparent performance monitors collected during a single run of the application.

Although studies have shown Peer Instruction (PI) in computing courses to be beneficial for learning and retention, study of the student experience has been limited to attitudinal survey results. This study provides a preliminary evaluation of student experiences in a PI course -- specifically asking them to reflect on their role as a student in a PI lecture compared to a standard university lecture. Student responses to this question are first analyzed using Chi's Interactive-Constructive-Active-Passive framework which categorizes student activities by their value in a constructivist learning framework. This analysis finds that the majority of students reported activity in a PI lecture as "interactive" in contrast with "active" (e.g. taking notes) in a standard lecture. Additionally, a grounded theory open-coding analysis provides an initial examination of student perceptions of the PI lecture experience. Although students positively value learning-related aspects (feedback and increased understanding) a surprising breadth of value was noted around issues of affect and increased sense of community. In particular, these experiences invite discussion about PI and issues of STEM retention in post-secondary education.

Peer Instruction (PI) has been shown to be successful at improving pass-rates and improving retention of majors in large classes at large research-intensive institutions. At these institutions, students have been shown to learn from peer discussion in PI and both students and faculty have reported that they value PI in their classrooms. However, little is known about the effectiveness of PI in small classrooms at teaching-focused liberal arts colleges. This study evaluates results from seven lower-division classes and four upper-division classes taught at three different liberal arts institutions using PI. In these classes, PI experienced similar success as that reported at large-research intensive universities, both in terms of student learning from peer discussion and from student attitudinal surveys. Most notably, of 137 surveyed students, 91% recommend more faculty use PI in their classes.

Many studies have demonstrated that students tend to learn less than instructors expect in CS1. In light of these studies, a natural question is: to what extent do these results hold for subsequent, upper-division computer science courses? In this paper we describe our work in creating high-level concept questions for an upper-division computer architecture course. The questions were designed and agreed upon by subject-matter and teaching experts to measure desired minimum proficiency of students post-course. These questions were administered to four separate computer architecture courses at two different institutions: a large public university and a small liberal arts college. Our results show that students in these courses were indeed not learning as much as the instructors expected, performing poorly overall: the per-question average was only 56%, with many questions showing no statistically significant improvement from pre-course to post-course. While these results follow the trend from CS1 courses, they are still somewhat surprising given that the courses studied were taught using research-based pedagogy that is known to be effective across the CS curriculum. We discuss implications of our findings and offer possible future directions of this work.

Peer Instruction (PI) is a teaching method that supports student-centric classrooms, where students construct their own understanding through a structured approach featuring questions with peer discussions. PI has been shown to increase learning in STEM disciplines such as physics and biology. In this report we look at another indicator of student success the rate at which students pass the course or, conversely, the rate at which they fail. Evaluating 10 years of instruction of 4 different courses spanning 16 PI course instances, we find that adoption of the PI methodology in the classroom reduces fail rates by a per-course average of 61% (20% reduced to 7%) compared to standard instruction (SI). Moreover, we also find statistically significant improvements within-instructor. For the same instructor teaching the same course, we find PI decreases the fail rate, on average, by 67% (from 23% to 8%) compared to SI. As an in-situ study, we discuss the various threats to the validity of this work and consider implications of wide-spread adoption of PI in computing programs.

Beginning in 2008, we introduced a new CS1 incorporating a trio of best practices intended to improve the quality of the course, appeal to a broader student body, and, hopefully, improve retention in the major. This trio included Media Computation, Pair Programming, and Peer Instruction. After 3 and 1/2 years (8 CS1 classes, 3 different instructors, and 1011 students passing the course) we find that 89% of the majors who pass the course are still studying computing one year later. This is an improvement of 18% over our average retention of 71% for the previous version of the course (measured since Fall 2001). If the focus shifts from retention of passing CS1 majors to retention of CS1 initially enrolled majors, multiple improvements--fewer students drop, more students pass, and more passing students are retained--compound to increase retention by 31% (from 51% to 82%). In this paper we analyze further aspects of these results, detail the three instructional design choices, and consider how they impact issues known to affect retention.

We introduce participants to Peer Instruction (PI): an active learning technique shown to be effective across the CS curriculum. In PI, Students work together to exchange perspectives and answer challenging conceptual questions, and are supported by short teaching segments. We will introduce and motivate PI, demonstrate its use in combination with a clicker system, and describe ways to encourage student preparation for PI classes. Pre-lecture preparation may include reading quizzes or exploratory homeworks, two topics of recent interest in the computing research literature. We will explore this literature as part of our tour of a complete PI course: from pre-lecture, to lecture, to the course at large.

Peer Instruction has recently gained interest in computing as an effective active learning pedagogy. The general focus of PI research has been on the in-class portion of PI: multiple choice questions and group discussion. Here, our focus is the reading quizzes completed by students for purposes of class preparation. These quizzes contain content questions but also ask for difficulties or confusion with course material. Consistent with expectations, we demonstrate that providing correct responses to quiz questions positively correlates with other course assessments. Somewhat counter-intuitively, we find that identifying confusions, noting problematic sections, or asking questions about the reading are also correlated with lab grades.

Peer Instruction (PI) is an active learning pedagogical technique. PI lectures present students with a series of multiple-choice questions, which they respond to both individually and in groups. PI has been widely successful in the physical sciences and, recently, has been successfully adopted by computer science instructors in lower-division, introductory courses. In this work, we challenge readers to consider PI for their upper-division courses as well. We present a PI curriculum for two upper-division computer science courses: Computer Architecture and Theory of Computation. These courses exemplify several perceived challenges to the adoption of PI in upper-division courses, including: exploration of abstract ideas, development of high-level judgment of engineering design trade-offs, and exercising advanced mathematical sophistication. This work includes selected course materials illustrating how these challenges are overcome, learning gains results comparing these upper-division courses with previous lower-division results in the literature, student attitudinal survey results (N = 501), and pragmatic advice to prospective developers and adopters. We present three main findings. First, we find that these upper-division courses achieved student learning gains equivalent to those reported in successful lower-division computing courses. Second, we find that student feedback for each class was overwhelmingly positive, with 88% of students recommending PI for use in other computer science classes. Third, we find that instructors adopting the materials introduced here were able to replicate the outcomes of the instructors who developed the materials in terms of student learning gains and student feedback.

We introduce participants to Peer Instruction (PI): an active learning technique applicable to the teaching of many subjects, including CS. In PI, Students work together to exchange perspectives and answer challenging conceptual questions, and are supported by short teaching segments. We will introduce and motivate PI, demonstrate its use in combination with a clicker system, and show that PI is much more than the use of clickers. Participants will work in groups to develop new PI questions addressing challenges to their students' learning, and discuss numerous pedagogical benefits conferred through PI.

The most significant source of lost performance when a thread migrates between cores is the loss of cache state. A significant boost in post-migration performance is possible if the cache working set can be moved, proactively, with the thread. This work accelerates thread startup performance after migration by predicting and prefetching the working set of the application into the new cache. It shows that simply moving cache state performs poorly, and that moving the instruction working set can be even more critical than data. This paper demonstrates a technique that captures the access behavior of a thread, summarizes that behavior into a compact form for transfer between cores, and then prefetches appropriate data into the new caches based on the summary. It presents a detailed study of single-thread migration effects, and then demonstrates its utility on a speculative multithreading architecture. Working set prediction as much as doubles the performance of short-lived threads, and in a full speculative multithreading implementation, the technique is also shown to nearly double the effectiveness of the spawned threads.

Peer Instruction (PI) is an instructional approach that engages students in constructing their own understanding of concepts. Students individually respond to a question, discuss with peers, and respond to the same question again. In general, the peer discussion portion of PI leads to an increase in the number of students answering a question correctly. But are these students really learning, or are they just "copying" the right answer from someone in their group? In an article in the journal Science, Smith et al. affirm that genetics students individually learn from discussion: having discussed a first question with their peers, students are better able to correctly, individually answer a second, conceptually-related question. We replicate their study, finding that students in upper-division computing courses (architecture and theory of computation) also learn from peer discussions, and explore differences between our results and those of Smith et al. Our work reveals that using raw percentage gains between paired questions may not fully illuminate the value of peer discussion. We define a new metric, Weighted Learning Gain, which better reflects the learning value of discussion. By applying this metric to both genetics and computing courses, we consistently find that 85-89% of "potential learners" benefit from peer discussion.

Peer Instruction (PI) has a significant following in physics, biology, and chemistry education. Although many CS educators are aware of PI as a pedagogy, the adoption rate in CS is low. This paper reports on four instructors with varying motivations and course contexts and the value they found in adopting PI. Although there are many documented benefits of PI for students (e.g. increased learning), here we describe the experience of the instructor by looking in detail at one particular question they posed in class. Through discussion of the instructors' experiences in their classrooms, we support educators in consideration of whether they would like to have similar classroom experiences. Our primary findings show instructors appreciate that PI assists students in addressing course concepts at a deep level, assists instructors in dynamically adapting their class to address student misunderstandings and, overall, that PI encourages students to be engaged in conversations which help build technical communication skills. We propose that using PI to engage students in these activities can effectively support training in analysis and teamwork skills.

Previous reports of a media computation approach to teaching programming have either focused on pre-CS1 courses or courses for non-majors. We report the adoption of a media computation context in a majors' CS1 course at a large, selective R1 institution in the U.S. The main goal was to increase retention of majors, but do so by replacing the traditional CS1 course directly (fully preparing students for the subsequent course). In this paper we provide an experience report for instructors interested in this approach. We compare a traditional CS1 with a media computation CS1 in terms of desired student competencies (analyzed via programming assignments and exams) and find the media computation approach to focus more on problem solving and less on language issues. In comparing student success (analyzed via pass rates and retention rates one year later) we find pass rates to be statistically significantly higher with media computation both for majors and for the class as a whole. We give examples of media computation exam questions and programming assignments and share student and instructor experiences including advice for the new instructor.

This research demonstrates that coming support for hardware transactional memory can be leveraged to significantly reduce the cost of implementing true speculative multithreading. In particular, it explores the path from eager conflict detection HTM to full support of efficient speculative multithreading, focusing on the case where frequent memory dependencies exist between speculative threads. The result is a unified memory architecture capable of effective support for transactional parallel workloads and efficient speculative multithreading.

Modern processors require highly accurate branch prediction for good performance. As such, a number of branch predictors have been proposed with varying size and complexity. This work identifies techniques to improve the accuracy of most predictors. It is especially effective with smaller, simpler predictors, allowing those predictors to be competitive with more expensive and complex variants.

Multi-core processors, with low communication costs and high availability of execution cores, will increase the use of execution and compilation models that use short threads to expose parallelism. Current branch predictors seek to incorporate large amounts of control flow history to maximize accuracy. However, when that history is absent the predictor fails to work as intended. Thus, modern predictors are almost useless for threads below a certain length.