Background and Context. The prominence of Generative AI (GenAI) has led students to seek academic support from these tools, altering the help-seeking landscape. Since GenAI can operate as an on-demand help resource, understanding the unique value of human staff can help instructors better serve students.

Background and Context. Problem decomposition is a core concern of computing education. It has also become increasingly relevant: in response to GenAI, many CS1 educators are advocating for shifting instructional emphasis away from code writing and towards decomposition and higher-level planning. Currently, there is a lack of knowledge in how novices do decomposition in large, multifunction tasks.

Peer code reviews involve students conducting a code review of a classmate's submission to a programming assignment. While peer code reviews have an established history of being used and studied in computing education, they have primarily been documented in introductory computing courses. This experience report describes how we implemented peer code reviews in an upper-division software engineering course that focuses on making modifications to large, existing code bases (i.e., brownfield development). We discuss the perceived learning benefits, perceived challenges, and agreement between peer and course staff reviews. Overall, students enjoyed being able to see different approaches to the programming task they had just submitted, but expressed concerns about feeling qualified to make effective peer code reviews due to their limited software engineering experience and the difficulty of assessing code design. We also find that students are capable of evaluating the functional correctness of their peer's submission, but struggle to give accurate assessments of their peer's design and code style. We conclude with recommendations specifically for instructors who wish to use peer code reviews in upper-division software engineering courses, such as using a structured template to scaffold the peer code reviews and allowing multiple opportunities to provide code reviews to improve students' self-efficacy.

The rising cost of learning resources creates obstacles to accessibility in higher education, particularly for students pursuing Computer Science. With the increasing availability and quality of open educational resources (OER), they have gained popularity as potential replacements for traditional paid materials. This study evaluates the effectiveness of an OER interactive textbook (eTextbook) and a paid eTextbook used in undergraduate computing courses. It focuses on students' perceived educational value of each, aiming to provide educators with insights that support informed decisions about course resources. We surveyed undergraduate students at a research-intensive US university who had taken an 'Intro to Programming' course with a free eTextbook and an 'Intro to Data Structures' course with a paid eTextbook to compare their effectiveness by asking students to rate the textbooks independent of the course difficulty. The collected data was analyzed to evaluate the comparative impact of free and paid eTextbooks. Our findings suggest that the OER eTextbook examined in this study can offer educational value comparable to or greater than the paid alternative. While the paid eTextbook was seen as more polished, students appreciated the free eTextbook's interactivity and course-specific customization, highlighting the importance of utility and engagement in learning resources. Although limited to the textbooks studied, our findings highlight the potential for OER to serve as a viable option in undergraduate computing courses. This insight may encourage instructors to consider OER as a cost-effective alternative that enhances accessibility without compromising educational quality.

Recent work has shown that incarcerated adult students reported a decrease in confidence in their ability to do well in the course as the course progressed, whereas non-incarcerated students in a traditional educational setting reported an increase in confidence over time on the same measure. Given these differences in student experiences between incarcerated adult students and traditional students, this work seeks to further understand the experiences of incarcerated adult in CS1, with a focus on their attitudes towards computing. Specifically, we used the Computing Attitude Survey (CAS) as a pre/post measurement in a CS1 course taught in prison. We found significant positive shifts for Problem Solving - Transfer and Fixed Mindset factors and a slight decrease for Real-World Connections. We additionally compare the results of the CAS survey between the incarcerated adult students in this study and those of non-incarcerated students reported in prior work.

Culturally Relevant Computing (CRC) has been shown to have positive effects on students, including increased classroom engagement, increased computing interest, and increased math performance. However, the vast majority of CRC studies take place in primary and secondary education, with little known about its efficacy in higher education. Despite the lack of literature on CRC in higher education, we believe that there may be CRC techniques presently being implemented in this context---yet not published in the research community. To investigate, we interviewed 21 professors from Hispanic Serving Institutions within the Computing Alliance for Hispanic Serving Institutions to document their implementation (if any) of CRC in the higher education context. Professors reported several CRC implementations including, but not limited to, culture in course materials, language-based approaches, providing opportunities outside the classroom, cultural sharing, and professional development. We conclude by discussing how these results may contribute to the promotion of leveraging students' cultures in higher education to serve students---particularly of minoritized backgrounds in computing.

Low-quality code can cost a company significant time and effort. As a result, code quality has been consistently studied in computing education research, especially in the context of CS1 students. However, less research has examined students' code quality while working on existing code bases (i.e., in tasks they are expected to do in industry). In this paper, we identify 1) common code quality defects introduced by upper-division students while contributing to an existing code base, 2) the severity, tool support, and language independence of those defects, and 3) programming experiences that may be associated with students' frequencies of defects, such as internship experience and use of Python (which was the language used in the programming tasks). In an upper division software engineering course, 48 students worked individually to 1) modify an existing feature and 2) implement a new feature in an open-source code base. Using an existing framework of code quality defects by Řechtáčková et al., we conducted a manual code review of all student submissions and found that students created defects related to Poor Design, Poor Documentation, Poor Formatting, and Unused Code at a high frequency. Students also seemed to copy-paste code from other files, which introduced defects related to Unused Code and Poor Design to their submission. Though our regression analysis did not reveal statistically significant predictors, students with prior internships, on average, introduced more code quality defects than those without any internship experience.

Program comprehension (PC) literature typically focuses on industry professionals comprehending large code bases or novice programmers comprehending short programs. As a result, limited work has aimed to understand how intermediate programmers comprehend large code bases, especially with the goal of supporting learners' incremental development of program comprehension expertise. Through the lens of the Block Model-a theory to support research on and teaching of PC-we aim to uncover 1) the comprehension process that intermediate programmers follow (i.e., top-down, bottom-up, etc.), and 2) common mappings between comprehension techniques used by intermediate programmers and comprehension blocks in the Block Model. We present a diary study of students' “process journals” inwhich they described their PC process while modifying the open-source idlelib code base. Our results showed that students typically followed a top-down and Text-first approach to understand a feature in the idlelib code base. Our findings also reveal how students used various program comprehension techniques (such as code navigation, using the IDE-based debugger, making experimental code changes, etc.) in terms of the Block Model. These findings make progress toward bridging our theoretical understanding of novices' comprehension process in small programs and expert's code comprehension process in large code bases by presenting a high sample size investigation of intermediate programmers' PC processes in a large, existing code base. Instructors can use our findings to understand which blocks in the Block Model are cover PC techniques, which can enable targeted teaching activities to impart PC skills.

Understanding students' testing processes in a CS1 course is crucial in helping instructors of introductory courses determine the necessary content to teach. Prior work highlights the importance of teaching testing practices to students, as there is concern for students' testing abilities upon graduation of an university CS program. Given that testing is an implicit programming process, we aim to examine how students in CS1 go about testing their code in programming assignments. Because of the consistent research showing the achievement gap between students with and without prior experience in introductory classes, our analysis also aims to understand specific differences in testing processes between the two groups. Leveraging a dataset of over 300 students with over 50,000 snapshots of student code during their development process, we applied metrics related to incremental testing and determined the usage of diagnostic print statements and the usage of designing test cases beyond the given tests (in which we refer to as ' custom test cases '). A large majority of the students used neither diagnostic print statements nor custom test cases in their programming assignments. Additionally, the three testing practices we examined do not seem to significantly contribute to the achievement gap due to prior experience to students' success, suggesting a need for further investigation into which practices do account for that success.

Understanding incarcerated adult (IA) students' fears upon entering a CS1 course and how their confidence changes throughout the course can help us understand how well IA students' fears are being addressed, and help future instructors of CS1 in prison address them better. Building on recent work on non-CS majors' fears and confidence in introductory CS, we surveyed 45 IA students across two offerings of a CS1 course in prison on their fears going into the course, and confidence in their ability to succeed. We present a phenomenographic analysis of fears amongst IA students in CS1, and analyze relationships between these fears and change in confidence. In addition, we compare the fears expressed by IA students to those of non-CS majors from prior work. Findings include many IA students reporting no fears, but an overall decrease in confidence across both offerings of the course which was mostly accounted for by students who did express initial fears. We found 9 fears overlapping with those found in a prior study outside of the prison context (e.g., getting a bad grade), and 7 fears only identified in our study (e.g., interference from circumstances beyond my control).

Large language models (LLMs) are already heavily used by professional software engineers. An important skill for new university graduates to possess will be the ability to use such LLMs to effectively navigate and modify a large code base. While much of the prior work related to LLMs in computing education focuses on novice programmers learning to code, less work has focused on how upper-division students use and trust these tools, especially while working with large code bases. In this study, we taught students about various GitHub Copilot features, including Copilot chat, in an upper-division software engineering course and asked students to add a feature to a large code base using Copilot. Our analysis revealed a novel interaction pattern that we call one-shot prompting, in which students ask Copilot to implement the entire feature at once and spend the next few prompts asking Copilot to debug the code or asking Copilot to regenerate its incorrect response. Finally, students reported significantly more trust in the code comprehension features than code generation features of Copilot, perhaps due to the presence of trust affordances in the Copilot chat that are absent in the code generation features. Our study takes the first steps in understanding how upper-division students use Github Copilot so that our instruction can adequately prepare students for a career in software engineering.

Higher education is expanding in United States prisons, with a growing demand for STEM offerings. Academics from other disciplines have stressed the importance of culturally relevant pedagogy (CRP) in prison higher education, and computing in context has shown major benefits in CS1--- especially for women and nontraditional students. More work is needed to determine what contexts are relevant to incarcerated college students, and how to incorporate these into computing curricula. In this paper, we build on prior work on computing in context and culturally relevant techniques in computing. We analyze course data from a CS1 course taught in a college-in-prison program to answer the following research question: What contexts do incarcerated students in CS1 find relevant? We identify 24 topics pursued by students across 78 open-ended programming assignment submissions, the three most popular being business management, sports statistics, and physical health. These results offer insight into potential contexts that are meaningful to incarcerated college students to be incorporated into future computing curricula and interventions in prisons.

In the United States, culturally relevant computing (CRC) is one of the most popular pedagogical implementations for Latin American (Latine) students. Culturally-relevant learning resources are a valuable tool for implementing CRC. However, the traditional method of creation and maintenance of textbooks takes a significant amount of time and effort. Given the duration required for textbook production, the development of culturally-relevant learning resources may become lengthened, as it requires close attention both on the material and the incorporation of cultural referents. In order to accelerate the process, we used the advancement of large language models (LLMs) to our advantage. Through prompt engineering, we created a series of prompts to produce a textbook for an introductory computer science course (CS1) that incorporates Latine culture. This textbook was evaluated on metrics regarding sensibility, correctness, readability, linguistic approachability, appropriateness of examples, and cultural relevance. Overall, the generated textbook was mainly sensible, correct, readable, and linguistically approachable. Code examples were not always appropriate due to the usage of libraries that are not typically used in a CS1 course. The cultural relevance was apparent, but it often included surface-level cultural referents. The main incorporation of culture was through geographical locations and people's names. This suggests that the use of LLMs to generate textbooks may serve as a valuable first step for writing culturally-relevant learning resources. Though this study focuses on Latines, our results and prompts may be applicable for generating culturally-relevant CS1 textbooks for other cultures.

Live coding is a recommended teaching practice in which an instructor dynamically programs in front of students. However, findings related to students' engagement during live coding are mixed. Some works have reported that live codingseems to improve student engagement while others regard live coding as an activity in which students passively observe the instructor without asking questions or following along. Active live coding, in which students extend a live coding example and discuss with peers, incorporates active learning with the traditional live coding approach. We conducted a quasi-experimental study in which one section of an advanced introductory programming course was taught using active live coding (ALC) and the other was taught using traditional live coding (TLC). The goal of this work is to compare students' behavioral engagement in the two lectures using a classroom observation protocol called the Behavioral Engagement Related to Instruction (BERI) protocol. Our results from the 2,790 observations we collected indicate that traditional live coding engages only 65% of students, on average. However, we found a "persisting engagement'' effect of active live coding, where students were significantlymore engaged in the traditional live coding components of a lecture up to 20 minutesafter the active live coding component. Notably, the two lecture groups performed similarly on the Post-Lecture Questions, which were administered after each lecture as a review of the lecture material. Therefore, our results indicate an improved student engagement due to active live coding, but do not show a corresponding improvement in conceptual knowledge.

Since March of 2020, universities around the world have offered remote versions of courses to help limit the spread of COVID-19. Two years later, in the Spring 2022 quarter, the lectures in the CS1 course at our large, public research-intensive university were taught via two modalities---an in-person modality in which students attended traditional, in-person lectures and a blended modality in which students attended a remote lecture on Zoom. Every other course component---labs, discussions, office hours---were held in-person for both groups. The unique setup of the CS1 course allowed us to perform a comparative analysis of the outcomes and attitudes between the two groups. In this paper, we analyze the difference in course outcomes, peer support, competitive feelings in class, and students' sense of belonging between the groups. Our results indicate that students in the blended learning group attended lectures more frequently than their in-person counterparts yet performed 4-7%worse on the midterm and final exams. The blended learning group also experienced significantlyless feelings of competitiveness than their in-person counterparts. Interestingly, we discovered a consistent trend among our results indicating that the gap in grades, peer support, and classroom competitiveness between the blended group and in-person group was more pronounced among first- and second-year undergraduates than third- and four-year students. Despite the two learning groups having different instructors, our results shed light on the potential advantages and drawbacks of a blended learning experience in CS1 that instructors should consider when deciding on the format of their course.

Efforts to bring incarcerated and formerly incarcerated individuals into the field of computing stand to improve equitable access to both computing jobs, and consequently the benefits of our tools and innovations through the inclusion of more diverse perspectives. This report describes the design and execution of a college level introductory computing course conducted with 26 students currently incarcerated at a prison in the United States in Fall 2022. We discuss the ways that the prison environment and the student body differ from traditional college computing classes, and how this impacted the design and execution of the course. We found that despite significant environmental barriers to learning to program, such as not having access to a code interpreter, there were unique affordances of the student population, including maturity and community, that could be leveraged in the course design and policies. We conclude with many lessons learned for the purpose of improving future offerings of computing courses in prisons.

Large language models (LLMs) like ChatGPT are changing computing education and may create additional barriers to those already faced by non-native English speakers (NNES) learning computing. We investigate an opportunity for a positive impact of LLMs on NNES through multilingual programming exercise generation. Following previous work with LLM exercise generation in English, we prompt OpenAI GPT-3.5 in 4 natural languages (English, Tamil, Spanish, and Vietnamese) to create introductory programming problems, sample solutions, and test cases. We evaluate these problems on their sensibility, readability, translation, sample solution accuracy, topicality, and cultural relevance. We find that problems generated in English, Spanish, and Vietnamese are largely sensible, easily understood, and accurate in their sample solutions. However, Tamil problems are mostly non-sensible and have a much lower passing test rate, indicating that the abilities of LLMs for problem generation are not generalizable across languages. Our analysis suggests that these problems could not be given verbatim to students, but with minimal effort, most errors can be fixed. We further discuss the benefits of these problems despite their flaws, and their opportunities to provide personalized and culturally relevant resources for students in their native languages.

Though there is a known issue with enrollment disparities across race/ethnicity in computing, U.S. Latin American (Latine) students have remained chronically underrepresented for decades. In order to actively support the recruitment, retention, performance, and experiences of U.S. Latines, we must make sure that we are carrying out studies with this population across outreach practices, interventions, student outcomes, and testimonies. To offer a more comprehensive understanding of the existing body of work on U.S. Latines in computing, we reviewed computing education papers that either fully center around or dedicate specific analyses to U.S. Latines. We analyzed 53 papers and found the following results: most research focuses on Student Experiences and Institutional Perspectives with very few studies on Research Opportunities; the educational setting tended to be K-12 with a severe lack of community college studies; there is a low number of popular pedagogical practices studied with U.S. Latines; research has mainly focused on students, but seldom focused families despite strong Latine connections to family; there was a strong spike in studies in 2021; and the U.S. location of the studies tended to line up with the U.S. Hispanic populations, but many states are under-performing. We discuss the implications of our findings and suggest future research directions to better understand, recruit, and support Latines in computing.

Cognitive Apprenticeship (CA) is an instructional model that outlines how experts can transfer their skills and knowledge to a learner for reasoning-based tasks, such as reading comprehension or mathematical problem solving. Specifically, CA includes 6 teaching methods---modeling, scaffolding, coaching, reflection, articulation, and exploration---that facilitate learners' observation, acquisition, and externalization of implicit processes and techniques for completing a task. In this paper, we present a systematic literature review of 143 conference papers across ACM and IEEE venues about CA in computer science education literature. Specifically, we aim to understand which teaching methods are typically referenced, the theory level (i.e., depth of CA theory discussion) present in the literature, and the key findings related to CA-based teaching approaches. Our review reveals that CA has been cited in computing education research as a guiding theory for various course designs, though there is a clear emphasis on papers related to modeling, scaffolding, and coaching whereas reflection, articulation, and exploration are under-explored. We found that CA methods have been effective in improving students' enthusiasm towards computing, improving pass-rates in courses, and improving instructors' capacity to accommodate more students by reducing instructor workload. However, a key challenge of CA approaches that emerged from our review is the difficulty in scaling the approach in settings with a high student to instructor ratio. Through this literature review, we aim to highlight effective CA approaches and how future initiatives can leverage CA to improve student learning.

Prior work has highlighted the gap between industry expectations for recent university graduates and the abilities those recent graduates possess. These works have even specifically recommended that students be given the opportunity to work on large, pre-existing code bases in their undergraduate career. This paper presents our experience teaching a newly-created course calledWorking with Large Code Bases. Guided by a Cognitive Apprenticeship approach to provide an authentic classroom experience that emphasizes the implicit processes and techniques involved in real-world software engineering, the course serves as a practical introduction to the skills and workflow involved in navigating and understanding a large code base. The goal of this experience report is to provide the motivation for key course design decisions, an overview of the course content, and a detailed description of key course components. We present student feedback indicating improved confidence in navigating a large code base and course outcomes related to specific tools and techniques students used in the course. Finally, we provide the full set of course materials we used and actionable recommendations for instructors to administer this course at their own institution, even with limited TA support.

ChatGPT is a conversational AI platform that can produce code to solve problems when provided with a natural language prompt. Prior work on similar AI models has shown that they perform well on typical intro-level Computer Science problems. However, little is known about the performance of such tools on Data Science (DS) problems. In this work, we assess the performance of ChatGPT on assignments from three DS courses with varying difficulty levels. First, we apply the raw assignment prompts provided to the students and find that ChatGPT performs well on assignments with dataset(s) descriptions and progressive question prompts, which divide the programming requirements into sub-problems. Then, we perform prompt engineering on the assignments for which ChatGPT had low performance. We find that the following prompt engineering techniques significantly increased ChatGPT's performance: breaking down abstract questions into steps, breaking down steps into multiple prompts, providing descriptions of the dataset(s), including algorithmic details, adding specific instructions to entice specific actions, and removing extraneous information. Finally, we discuss how our findings suggest potential changes to curriculum design of DS courses.

Background and Context. Live coding is a teaching method in which an instructor dynamically writes code in front of students in an effort to impart skills such as incremental development and debugging. By contrast, traditional, static-code examples typically involve an instructor annotating or explaining components of pre-written code. Despite recommendations to use live coding and a wealth of qualitative analyses that identify perceived learning benefits of it, there are a lack of empirical evaluations to confirm those learning benefits, especially with respect to students’ programming processes.

Online discussion boards, designed to facilitate learning from peers and instructors in an accessible space, are a vital part of course design, especially in large scale computer science classes. Previous work has shown that women in computer science tend to use anonymity more often than men on these boards, a trend not found in humanities, social science or business courses. In this work, we build on these findings using an intersectional lens, analyzing both gender and race/ethnicity. We find this combined analysis reveals differences in anonymity that are not apparent when examining gender alone. For example, we find a significantly greater difference in anonymity use between Hispanic men and women than would be expected from analyzing race/ethnicity and gender independently. We additionally analyze type of content (e.g., questions, answers), course, platform, and data source to characterize the many factors at play in measuring students’ choice to participate anonymously. In doing so, we show that different approaches used in prior work for eliciting information on gender — whether using registrar data, a survey, or imputing gender based on name — changes how over of students are classified, particularly affecting nonbinary students and Asian students. Understanding when students participate anonymously can help educators and platform designers to make students’ experience of online discussion boards more welcoming.

Prior studies in multilingual computing education have shown that many non-native English speakers (NNES) in India struggle with introductory programming courses as they learn both a programming language (e.g., Java) and a natural language (e.g., English) concurrently. Although multiple studies have been conducted with NNES in India whose first language is Hindi or Tamil, we do not yet know the influence a students' native language may have among Spanish speaking students in the United States. This replication study investigates the effects of an instructional design integrating the students' native language along with English on high school students' learning and engagement in a two week CS0 course using the block-based programming language, Scratch. We designed an experiment to teach introductory computing topics (e.g., algorithms, variables, loops, conditionals) to two groups of students from a rural area spanning multiple institutions in the US. The experimental group was taught using English and Spanish (students' native language) and the control section was taught using only English. A pre-test and post-test was conducted to test students' programming knowledge before and after the course. We also recorded all the questions students asked during the course to measure student engagement. We found that teaching Scratch programming using Spanish and English is no different than teaching Scratch programming using only English to high school students whose native language is Spanish. We also found that the students in the experimental group asked more questions when compared to the control group.

Live coding---a pedagogical technique in which an instructor plans, writes, and executes code in front of a class---is generally considered a best practice when teaching programming. However, only a few studies have evaluated the effect of live coding on student learning in a controlled experiment and most of the literature relating to live coding identifies students' perceived benefits of live-coding examples. In order to empirically evaluate the impact of live coding, we designed a controlled experiment in a CS1 course taught in Python at a large public university. In the two remote lecture sections for the course, one was taught using live-coding examples and the other was taught using static-code examples. Throughout the term, we collected code snapshots from students' programming assignments, students' grades, and the questions that they asked during the remote lectures. We then applied a set of process-oriented programming metrics to students' programming data to compare students' adherence to effective programming processes in the two learning groups and categorized each question asked in lectures following an open-coding approach. Our results revealed a general lack of difference between the two groups across programming processes, grades, and lecture questions asked. However, our experiment uncovered minimal effects in favor of the live-coding group indicating improved programming processes but lower performance on assignments and grades. Our results suggest an overall insignificant impact of the style of presenting code examples, though we reflect on the threats to validity in our study that should be addressed in future work.

Research efforts tried to expose students to security topics early in the undergraduate CS curriculum. However, such efforts are rarely adopted in practice and remain less effective when it comes to writing secure code. In our prior work [18], we identified key issues with the how students code and grouped them into six themes: (a) Knowledge of C, (b) Understanding compiler and OS messages, (c) Utilization of resources, (d) Knowledge of memory, (e) Awareness of unsafe functions, and (f) Understanding of security topics. In this work, we aim to understand students' knowledge about each theme and how that knowledge affects their secure coding practices. Thus, we propose a modified SOLO taxonomy for the latter five themes. We apply the taxonomy to the coding interview data of 21 students from two US R1 universities. Our results suggest that most students have limited knowledge of each theme. We also show that scoring low in these themes correlates with why students fail to write secure code and identify possible vulnerabilities.

Previous research has been devoted to improving the experience of non-majors in introductory CS courses. In this study, we compare the experiences of non-majors in two different introductory CS courses, specifically with respect to fears about taking the course and change in confidence levels. CS0 is a computing course intentionally designed for non-majors, and CS1 is a more traditional introductory computing course. Both of these courses were composed primarily of non-majors and were taught by the same instructor. Survey data was collected from 124 students enrolled in CS0, and 502 students enrolled in CS1. Through qualitative analysis, we found that the fears of non-major students entering both of these introductory CS courses fell into one or more of nine distinct categories (e.g., Coding, Perceiving STEM as Difficult, Managing Workload). Additionally, using students' confidence levels at the beginning and end of the courses, we found that students in CS0 had a greater increase in confidence level than those in CS1. Finally, we explored connections between students' fears and how their confidence changed by the end of the course. We found that students across both courses with fears related to coding, lack of preparation, and being left behind had the highest average increase in confidence levels.

Incremental development is the process of writing a small snippet of code and testing it before moving on. For students in introductory programming courses, the value of incremental development is especially higher as they may suffer from more syntax errors, lack the proficiency to address complicated bugs, and may be more prone to frustration when struggling to correct code. However, to evaluate the effectiveness of interventions that aim to teach programming processes such as incremental development, we need to develop measures to assess such processes. In this paper, we present a way to measure incremental development. By qualitatively analyzing 15 student coding interviews, we identified common behaviors in the programming process that relate to incremental development. We then leveraged a dataset of over 1000 development sessions -- about 52,000 code snapshots at compilation time -- to automatically detect the common behaviors identified in our qualitative analysis. Finally, we crafted a formal metric, called the "Measure of Incremental Development'' (MID), to quantify how effectively a student used incremental development during a programming session. The MID detects common non-incremental development patterns such as excessive debugging after large additions of code to automatically assess a sequence of snapshots. The MID aligns with human evaluations of incrementality with over 80% accuracy. Our metric enables new research directions and interventions focused on improving students' development practices.

Previous studies have shown that non-native English speakers (NNES) have lower confidence levels at the beginning of the term and higher academic expectations for themselves throughout the term. Additionally, although non-native English speakers study for longer hours compared to native English speakers (NES), there were not any differences in their grades. To better understand the experiences of NNES learning Computer Science in a U.S university, we examined possible factors that may contribute to differences in stress and sense of belonging between NES and NNES. From our investigation using weekly surveys, we did not find any significant difference in the stress level reported by NES and NNES. However, we found differences in the factors that contribute to the stress level of NES and NNES. Factors such as 'personal life' and 'requirements for other classes' were major factors that contributed towards the stress level of NES, whereas 'embarrassment asking for help' and 'self doubt' were more common among NNES. As for sense of belonging, we found no significant difference between NES and NNES. Additionally, we found that the number of classmates that NES feel comfortable reaching out to is not a factor affecting their sense of belonging in an introductory computer science class. \endabstract

Novice programmers often struggle with code understanding and debugging. Live Programming environments visualize the runtime values of a program each time it is modified to provide immediate feedback, which help with tracing the program execution. This paper presents the use of a Live Programming tool in a CS1 course to better understand the impact of Live Programming on novices' learning metrics and their perceptions of the tool. We conducted a within-subjects study at a large public university in a CS1 course in Python (N=237) where students completed tasks in a lab setting, in some cases with a Live Programming environment, and in some cases without. Through post-lab surveys and open-ended feedback, we measured how well students understood the material and how students perceived the programming environment. To understand the impact of Live Programming, we compared the collected data for students who used Live Programming with the data for students who did not. We found that while learning outcomes were the same regardless of whether Live Programming was used or not, students who used the Live Programming tool completed some code tracing tasks faster. Furthermore, students liked the Live Programming environment more, and rated it as more helpful for their learning.

Often, security topics are only taught in advanced computer science (CS) courses. However, most US R1 universities do not require students to take these courses to complete an undergraduate CS degree. As a result, students can graduate without learning about computer security and secure programming practices. To gauge students' knowledge and skills of secure programming, we conducted a coding interview with 21 students from two R1 universities in the United States. All the students in our study had at least taken Computer Systems or an equivalent course. We then analyzed the students' approach to safe programming practices, such as avoiding unsafe functions like gets and strcpy, and basic security knowledge, such as writing code that assumes user inputs can be malicious. Our results suggest that students lack the key fundamental skills to write secure programs. For example, students rarely pay attention to details, such as compiler warnings, and often do not read programming language documentation with care. Moreover, some students' understanding of memory layout is cursory, which is crucial for writing secure programs. We also found that some students are struggling with even the basics of C programming, even though it is the main language taught in Computer Systems courses.

One of the goals of computing education research is to document the potential strengths and weaknesses of contemporary teaching methods in computing. Live coding has recently gained attention as one of the best practices for teaching programming. To offer a more comprehensive understanding of the existing body of research about live coding, we reviewed papers in computing education research that investigated the value of live coding in an educational setting. We categorized each paper based on (1) how it defines live coding, (2) whether its version of live coding could be considered active learning, (3) the type of study conducted, (4) types of data collected and the data analysis methods used, (5) evidence provided for the effectiveness of live coding, (6) reported benefits and drawbacks of live coding, and (7) reported theoretical frameworks used to explain the basis, effects or goals of live coding. We found that although live coding has been recommended as one of the best practices for teaching programming, there is a lack of empirical evidence to support claims about the effectiveness of live coding on student learning. Finally, we discuss the implications of our findings and suggest future research directions that could develop a more holistic understanding of this pedagogical technique.

Introductory computer science courses, such as Computer Systems, could be used to provide the first exposure to computer security to students. However, prior work has shown that, in the US's top R1 universities, computer systems courses are not taught with security in mind. It was also shown that students and instructors use unsafe functions in their code, leading to security vulnerabilities. In this paper, we focused on the textbooks used for computer systems courses. We analyzed the discussion of security topics and the use of unsafe functions in the thirteen textbooks used in the top 30 R1 universities in the US for teaching computer systems. We show that many textbooks do not discuss security at all, while some limit their discussion to "undefined behavior'', ignoring that opportunity to discuss potential security issues associated with the undefined behavior. Furthermore, textbooks that talk about security continue using unsafe functions throughout (though not necessarily in vulnerable ways but also without any warning or explanation). We also show that many textbooks do not warn about unsafe functions they use or teach how to use them safely.

Prior work has shown that the adjustment challenges faced by international students in English-speaking universities are primarily attributable to English language proficiency and culture. Although studies in computing education have documented the barriers that non-native English speakers face while learning CS using online programming platforms, little research has been done on their experiences in CS courses within a university setting. In this study, we seek to understand the experiences of non-native English speakers learning CS at a English-speaking university in the United States. Through weekly surveys and course grades from four CS courses, we investigate the differences between non-native and native English speakers with respect to the time spent studying outside of class, confidence levels at the start of a course, types of learning resources used, and expected and actual overall grades. We found that although non-native speakers require a higher minimum grade to be satisfied and often spend a longer time on average studying outside of class compared to native speakers, there was no difference in the actual grade received between the two groups. While there was no significant difference in the type of resources used (online vs in-person), there was a significant difference in the confidence level between native and non-native English speakers.

Introductory computer systems courses teach students how a single program is executed inside a computer, providing them with their first exposure to the logical internals of computing systems. This is one of the first introductory courses where students can learn about security and the need for robust coding. However, currently, these courses are taught with a focus on functionality and efficiency only, ignoring security almost entirely.

Understanding an operating systems (OS) code base is a difficult task since it involves understanding a huge amount of low-level C and assembly code. The inherent level of difficulty associated with OS topics is high because of the high element interactivity (i.e., material consists of elements that heavily interact). The mental effort associated with learning a complex subject like OS may be higher for non-native English speakers, when the subject is taught in a natural language (i.e., English) that is not the students' native language. We were interested in finding the effect of an instructional design that combines the students' native language along with English on students' understanding of select topics in OS. We designed an experiment to teach CPU virtualization using xv6 to two groups of undergraduate students in Tamil Nadu, India. We taught the experimental group using English and Tamil (native language of students in Tamil Nadu) and the control group using only English. We conducted a pre-test and a post-test to test students' understanding of the OS topics taught, before and after our intervention respectively. We also collected data on the questions that students asked in lectures during our intervention. We found that teaching OS using native language and English is no different than teaching OS using only English with respect to student learning. We also found that the native language had an impact on the student engagement and classroom interaction by creating more dialogue within the Tamil+English (experimental) classroom when compared to the English-only (control) classroom.

Computer Science (CS) is taught in India, using English as the medium of instruction, to students whose native language is not English. This places a high cognitive load on students who learn programming for the first time and who are not very proficient in English. The problems these students face become even harder since learning to program can be an incredibly difficult task. Our study aims to find out if a student's native language has any effect on the student's ability to learn programming. We taught linked list, a basic concept in CS, to two groups of undergraduate students for a week in Tamil Nadu, India. We used English to teach one group of students and English and Tamil (the native language in Tamil Nadu) to teach the other group. Our intervention consisted of three lectures and one live-coding session. We collected quantitative and qualitative data using technical tests and open-ended feedback respectively. We found that although teaching programming using both English and Tamil is no different when compared to using only English with respect to student learning, students have expressed positive opinions about teaching and learning CS using two languages.