Researchers use replication to confirm, strengthen, and advance Computing Education Research (CER). However, prior research shows that replication is infrequently used in CER, even though the community encourages its use. Previous research suggests that the CER community uses different terms to describe replication, which we aim to confirm in this Working Group (WG). We will conduct a Systematic Literature Review (SLR) across influential international CER venues to understand how researchers present and conduct replication studies, possibly identifying venues receptive to papers applying this research design. In addition, we will interview Computing Education researchers, conference leaders, and journal editors to understand their experiences and perceptions with replication. We expect to collect suggestions and recommendations on how CER can encourage more replication in future studies. Our work will highlight and confirm the terms the CER community uses to present replication studies, enabling researchers and educators to better identify these studies.

Previous work on prerequisites in computing found that GPAs correlated more strongly with course grades than direct measures of prerequisite knowledge. This suggests that course grades encompass more than subject knowledge. This work was conducted in an advanced data structures (ADS) course at a single institution. To investigate the generalizability of these findings, we replicate the study in an upper-year machine learning course. We find that the pattern holds. These results underscore the need to distinguish course performance from demonstrated skill.

In this session, we aim to foster a community among computer science faculty across 2-year and 4-year primarily undergraduate institutions (PUIs). We will foster an inclusive environment through activities that allow for multiple forms of engagement (e.g. shows of hands, small group introductions, and full group discussion). We will facilitate discussions about ways computer science is changing at PUIs, differences seen across institutions, and ways that our community can be sustained outside of the SIGCSE TS. In prior years, discussions have focused on topics such as availability of course staff and tutors, availability of conference funding, organizational infrastructure such as sponsored program offices, and requirements for promotion.

In a field as interdisciplinary as Computer Science Education, navigating the landscape of research paradigms and methodologies can be both exciting and overwhelming. Research design decisions are often constrained by time, disciplinary familiarity, and uncertainty about the benefits of alternative approaches. In Computer Science Education Research—an inherently interdisciplinary field—these constraints can prevent researchers from exploring the full range of possibilities available to them. Cards for Alternative Research Design (CARD) is designed to expand researchers' awareness of different research approaches through a card-based research design activity. By engaging with paradigms, methodologies, and constraints in a playful format, participants can critically reflect on their current research without the formality or potential confrontation of direct critique. This exercise strengthens the reasoning behind chosen approaches, while also surfacing valid arguments for adopting alternatives, all with a relatively low time investment.

In this session, we aim to foster a community of computer science faculty across many primarily undergraduate institutions. We will foster an inclusive environment through activities that allow for multiple forms of engagement (e.g. shows of hands, small group introductions, and full group discussion). We will facilitate discussions about ways computer science is changing at PUIs and continue to brainstorm ways that our community can be sustained outside of the SIGCSE TS.

Undergraduate teaching assistants (tutors) are commonly employed in computing courses to help students with programming assignments. Prior research in computing education has reported the benefits of tutoring both for students and for the tutors' own learning. In contrast, recent research that examined actual tutoring sessions has reported that these sessions may be less productive than one might hope, with tutors often just giving students the answers to their problems without trying to teach the underlying concepts. To better understand why tutors may be employing these suboptimal practices, we interviewed ten tutors across early computing courses in higher education to identify their perceived role in these sessions, what stressors and factors influence their ability to perform their job effectively, and what kinds of best practices they learned in their tutor training course. Tutors reported their roles around student learning, gauging student understanding, identifying or providing solutions to students, and providing socioemotional support. They reported their stressors around environmental factors (e.g., number of students waiting to be helped, preparation time, peer-tutor frustrations), internal influences, student behavior, student skill levels, and feeling the need to ''read a student's mind.'' Regarding their tutor training course, Tutors reported learning about interaction guidelines and procedures and question-based problem solving. We conclude by discussing how these results may contribute to the less-effective behaviors seen in prior research and potential ways to improve tutoring in computing courses.

We present results of an intervention integrating a tool promoting metacognitive study behaviors, CompassX, in a Data Structures and Algorithms (CS2) course. Metacognition---commonly referred to as ''thinking about thinking''---has been consistently linked to improved learning strategies and student achievement. However, no prior literature has practiced an intervention that addresses all three commonly-accepted phases of metacognition. Thus in this work, we share the key features of CompassXthat promote metacognitive study behaviors, how our users engaged with those features, and how continued practice of metacognition using those features is related to improved learning outcomes. Students used CompassX voluntarily and some students did not fully engage with all metacognition-based features. However, continued engagement with a metacognitive feature appears to be indicative of higher exam scores. This also implies the possibility of utilizing a metacognitive tool to improve the performance of student outcome modeling by collecting a new type of behavioral data.

In this session, we aim to foster a community of computer science faculty across many primarily undergraduate institutions. We will foster an inclusive environment through activities that allow for multiple forms of engagement (e.g. shows of hands, small group introductions, and full group discussion). We will facilitate discussions about ways computer science is changing at PUIs and brainstorm ways that our community can be sustained outside of the SIGCSE TS.

In this Birds of a Feather (BoF) session, we aim to create and foster a safe and open environment that allows graduate students share experiences and issues with graduate studies in CSEd research. We invite genuine and honest conversations about the challenges facing rising academics and professionals in this area. We have three goals: (1) to foster a community of support among CSEd grad students from around the world, (2) to connect students with new opportunities and ways of thinking about CSEd, and (3) to cultivate mentorship among participants and send them off with potential collaborators, supporters, and advocates.

Recent research in computing has shown that student performance on prerequisite course content varies widely, even when students continue to progress further through the computing curriculum. Our work investigates instructors' perspectives on the purpose of prerequisite courses and whether that purpose is being fulfilled. In order to identify the range of instructor views, we interviewed twenty-one computer science instructors, at two institutions, that teach a variety of courses in their respective departments. We conducted a phenomenographic analysis on the interview transcripts, which revealed a wide variety of views on prerequisite courses. The responses shed light on various issues with prerequisite course knowledge, as well as issues around responsibility and conflicting pressures on instructors. These issues arise at the department level, as well as with individual course offerings.

Many institutions use undergraduate teaching assistants (tutors) in their computing courses to help provide more resources to students. Because of the role tutors play in students' learning experiences, recent work in computing education has begun to explore student-tutor interactions through the tutor's perspective and through direct observation of the interactions. The results suggest that these interactions are cognitively challenging for tutors and may not be as beneficial for students' learning as one might hope. Given that many of these interactions may be unproductive, this work seeks to understand how student expectations of these sessions might be impacting the interactions' effectiveness. We interviewed 15 students in a CS2 course to learn about the expectations and desires that students have when they attend tutoring sessions. Our findings indicate that there is variation in what students consider a desired result from the interaction, that assignment deadlines affect students' expectations and desires for interactions, and that students do not always want what they believe is beneficial for their learning. We discuss implications for instructors and potential guidance for students and tutors to make tutoring sessions more effective.

As enrollments in computing courses have surged, the ratio of students to faculty has risen at many institutions. Along with many other large undergraduate programs, our institution has adapted to this challenge by hiring increasing numbers of undergraduate tutors to help students. In early computing courses, their role at our institution is primarily to help students with their programming assignments. Despite our institution offering a training course for tutors, we are concerned about the quality and nature of these student-tutor interactions. As instruction moved online due to COVID-19, this provided the unique opportunity to record all student-tutor interactions (among consenting participants) for research. In order to gain an understanding of the behaviors common in these interactions, we conducted an initial qualitative analysis using open coding followed by a quantitative analysis on those codes. Overall, we found that students are not generally receiving the instruction we might hope or expect from these sessions. Notably, tutors often simply give students the solution to the problem in their code without teaching them about the process of finding and correcting their own errors. These findings highlight the importance of tutoring sessions for learning in introductory courses and motivate remediation to make these sessions more productive.

In this Birds of a Feather (BoF) session, we aim to create and foster a safe and open environment that allows participants to get feedback on research-in-progress and share experiences and issues with being a graduate student in CSEd research. We invite genuine and honest conversations about the challenges facing rising academics and professionals in this area. We have three goals: (1) to foster a community of support among CSEd grad students from around the world, (2) to connect students with new opportunities and ways of thinking about CSEd, and (3) to cultivate mentorship among participants to send them off with potential collaborators, supporters and advocates.

Previous work in computing has shown that Black, Latinx, Native American and Pacific islander (BLNPI), women, first-generation, and transfer students tend to have worse outcomes during their time in university compared to their majority counterparts. Previous work has also found that students' incoming prerequisite course proficiency is positively correlated with their outcomes in a course. In this work, we investigate the role that prerequisite course proficiency has on outcomes between these groups of students.

Students’ sense of belonging has been found to be connected to student retention in higher education. In computing education, prior studies suggest that a hostile culture and a feeling of non-belonging can lead women, Black, Latinx, Native American, and Pacific Islander students to drop out of the computing field at a disproportionately high rate. Yet, we know relatively little about how computing students’ sense of belonging presents and evolves (if at all) through their college courses, particularly in courses beyond the introductory level, and little is known about how sense of belonging impacts student outcomes in computing. In an extension of a previous study, we examined students’ sense of belonging in six early undergraduate computer science courses across three consecutive quarters at a large research-intensive institution in North America. We found that women and first generation students have a lower incoming sense of belonging across all courses. When exploring sense of belonging’s tie to student outcomes we found that lower sense of belonging was correlated with negative course outcomes in terms of pass rates and course performance. We also found that it is less tied to student performance as students get further into the CS curriculum. Surprisingly, there was no indication that sense of belonging is predictive of retention in terms of persistence to the next CS course outside of the first course in our two-course CS1 sequence.

Although Massive Open Online Courses have the potential to reach a much broader audience and offer a lower cost education than traditional in-person classes, they have struggled with low completion rates and low diversity amongst those enrolled and completing the courses. In 2015, we built a series of online courses in computing with the specific goal of attracting and retaining students from groups underrepresented in computing. In our design, we incorporated a number of features aimed at improving the inclusive nature of the courses including: a project-centered course design; an online version of Peer Instruction ConceptTests; videos where students, faculty, and professionals report their struggles when they first learned computing concepts; videos by professional software engineers explaining how computing concepts from the course are used in industry; and videos aimed at providing additional support on the project to students who might be struggling. In this work, we report on the design of the courses and examine how successful our courses were at attracting and retaining women students. We find that compared to other computing courses offered by our institution on the same platform, our courses have: a higher percentage of women enrollment, higher rates of course completion for both men and women, and a slightly smaller gap between completion rates for men and women.

Spring 2020 brought enormous change to student learning, as universities scrambled to put into place support structures to aid students' learning in a remote context. Computer science education was both well-positioned for this change and faced unique challenges, e.g. that students often need significant (in-person) support with programming. In this study we examine how aspects of students' remote learning experience in spring 2020 compared to the same aspects in previous in-person, pre-COVID offerings of 6 lower- to mid-division computer science courses at UC San Diego (UCSD), a large US research university. We were in a unique position to make this comparison because we had been collecting data on several aspects of students' course experiences throughout the 2019-2020 academic year. We found, surprisingly, that most elements of students' experiences that we examined were unchanged, or even improved, in spring 2020. Students in spring reported similar or lower stress levels and found their courses similarly or less challenging relative to previous quarters. However, some aspects did degrade. Students had less connection with their peers (particularly in introductory classes), more interference from family obligations, and higher drop/fail rates in some classes. Surprisingly, these results hold across all assessed demographics. Our results indicate that the actions UCSD and its CS instructors took to mitigate the stresses of remote learning in spring 2020 were largely successful and provide implications for improving education beyond the pandemic.

Previous studies show that CS students may not learn as much from their courses as we might expect. This could have ramifications on how students succeed in their future careers and may explain why researchers report a gap between industry expectations and the abilities of recent CS graduates. However, previous studies have also shown that students improve their prerequisite knowledge in subsequent courses. This study investigates the introductory data structures proficiency of students in different courses at various stages in our CS program, employing the validated Basic Data Structures Inventory (BDSI). Additionally, we investigate whether subpopulations, including transfer students and underrepresented groups, may be more prone to not attaining as much knowledge from our courses as we might expect. We find that students' knowledge of basic data structures is, on average, better in later courses. However, we also find subpopulations of students that perform worse than others or seem to not improve their knowledge in later courses. Specifically, we find students that transferred to our institution from a different school perform significantly worse on the BDSI than other students and do not improve their BDSI performance in later courses. We also find students from demographic backgrounds that are underrepresented in computing scored slightly, though not statistically significantly, worse than others. Our findings warrant future investigations on how our programs can better serve the students in the affected subpopulations.

In this Birds of a Feather (BoF) session, we aim to create and foster a safe and open environment that allows participants to get feedback on research-in-progress and share experiences and issues with being a graduate student in CSEd research. We invite genuine and honest conversations about the challenges facing rising academics and professionals in this area. We have three goals: (1) to foster a community of support among CSEd grad students, (2) to connect students with new opportunities and ways of thinking about CSEd, and (3) to cultivate mentorship among participants to send them off with potential collaborators, supporters and advocates.

Although elements of the academia-industry gap have been studied extensively, these studies have mostly ignored the primary stakeholder for changing academia: faculty. Building on a recent qualitative study that revealed a wide range of faculty views on the gap, this study quantitatively examines faculty views through a survey on the goals of CS education, how CS programs should address the academia-industry gap, and which barriers prevent adoption of remedies. Analysis of the 249 responses reveals that a majority of faculty share common goals in supporting student preparation for a career in industry. Moreover, faculty strongly view their own institutions as the prime party responsible for student preparation for careers in both academia and industry. We also find that whereas faculty are generally in agreement on what could be improved to provide students with better industry preparation, some reported far greater barriers to implementing those improvements than others.

Given the long-standing concern about students failing introductory programming courses, there is a need for interventions that may aid those students. In this work, we examine the potential benefit of three interventions based on prior computing education research (CER) or STEM education research literature: mindset interventions, the use of "Thinkathons" as an alternative to programming labs, and metacognitive interventions to encourage more productive study habits. We conducted an in-class study that controlled for both time-on-task and selection bias to investigate the potential benefits of integrating these interventions into the existing footprint of an introductory computing course. Despite the previously reported promise of the interventions we implemented, our findings were that in this context these techniques had only a mild positive effect for some students. We discuss possible reasons why these techniques are less successful than instructors might hope and argue for the need for more research on this topic.

Recent work has identified a mismatch between instructor expectations of students' mastery of prerequisite course content and their actual ability. This invites the question of why this mismatch exists. We first examined grades in prerequisite courses and found they meaningfully correlated with performance on an assessment testing their knowledge of prerequisite material. In addition, we found neither taking alternatives to the primary identified prerequisites nor the delay between taking prerequisite courses and the follow-on course meaningfully impacts performance. Second, we confirmed that prerequisite course grades are significantly correlated with the grade in the follow-on course-confirming that the grades in the previous courses convey some information about student understanding of those topics. Perhaps surprisingly, we found that grades in courses outside computing were similarly correlated as those courses inside computing, suggesting that underlying factors such as general study skills may be as important as the domain-specific knowledge itself.

Previous work has found that recent computer science graduates often experience difficulty transitioning into their new roles in industry due to a significant gap between their academic experiences and industry's expectations. Although multiple studies have identified the views of students and members of industry on the value of a CS degree as preparation for industry, the faculty perspective on this topic remains unclear. Understanding these views could shed light on why the academia-industry gap has persisted despite the attention. This study identified faculty views on the goals of an undergraduate education and a CS major, focusing on preparation for careers in industry. In order to identify a spectrum of faculty views, we interviewed 14 faculty from a variety of backgrounds across three institutions. A phenomenographic analysis of the transcripts reveals that many faculty believe that industry preparation is an important programmatic goal, yet they encounter significant resource obstacles to achieving that goal.