The rise of generative artificial intelligence (GenAI) has sparked a rapid change in computing curricula and teaching approaches. GenAI coding tools can accurately complete assignments, answer test questions, and perform other tasks traditionally associated with learning programming, especially at the introductory level. Because GenAI is still so new, researchers investigating student usage of GenAI have used informal rubrics and questionnaires. To advance, the field needs validated instruments that measure student perception and use of GenAI. This paper presents the development and initial validation of an instrument to measure self-efficacy while using GenAI to learn programming. Self-efficacy is an important construct in education research because it robustly correlates with student success, across disciplines and ages, including undergraduate computing education. Computing education researchers have presented several validated self-efficacy instruments, most recently by Steinhorst et al. in 2020. Critically, this instrument was created before the rise of GenAI’s popularity in 2022. To complement this instrument, we created a GenAI scale similar in style to the Steinhorst self-efficacy instrument, consisting originally of 11 items and revised to 5 items. We report two important findings in this paper. First, we found strong support for the validity of the existing Steinhorst instrument in a new context, specifically an introductory programming course that fully integrates GenAI. Second, the new GenAI scale shows strong internal reliability, discriminant validity with items in the Steinhorst subscales, and criterion validity with students’ GenAI usage patterns. Based on statistical analysis and cognitive probing interviews, we argue for the validity of the five-item scale to measure students’ GenAI self-efficacy in the context of programming.

Modern programming tools use generative AI (GenAI) to suggest code to the user as they type, interrupting their problem-solving behavior and undermining the development of their programming critical thinking skills. In this paper, we present a scaffolded programming exercise designed to support student differentiation between good and bad GenAI code suggestions based on negative expertise–that identifying why an answer is wrong is part of developing conceptual knowledge. We compare a version of the tool that showed one suggestion (correct or not), to a version that showed three suggestions (one of which was correct). We present results on performance and error rates as well as qualitative findings centered on Pintrich and DeGroot’s theory of self-regulation. Students reported that the single suggestion version better aligned with industry tools and presented a lower cognitive load. Students also reported that the multiple suggestion version caused them to slow down and think critically about the line under consideration, the overall purpose of the code, and the benefits of planning.

Introductory programming courses present a unique challenge for many students as a novel discipline, requiring significant time investment and featuring a steep learning curve, resulting in students experiencing high levels of failure while learning. Students' perspectives on failure are crucial in determining how they confront these challenges and, consequently, their learning outcomes. This study investigates the relationship between undergraduate students' views on failure--measured by validated scales about growth mindset, fear of failure, self-efficacy, and academic resilience--with their performance in introductory programming courses. While self-efficacy and growth mindset are well-studied in computing education, fear of failure and academic resilience remain understudied despite their prominence in other disciplines. We collected data from three universities to conduct a repeated measures study of 58 students' attitudes toward failure at the beginning and the end of the semester. Our results indicated self-efficacy and fear of failure uniquely predicted performance, with lower self-efficacy and higher fear of failure related to poorer outcomes. Furthermore, students with lower self-efficacy and higher fear of failure were four times more likely to withdraw from or fail the course. Our findings suggest that measuring self-efficacy and fear of failure at the beginning of the semester can help identify at-risk students who need support. Research and interventions related to academic fear of failure from other STEM fields should be examined in the context of computing education to improve outcomes for our students.

Integrated computing uses computing tools and concepts to support learning in other disciplines while giving all students opportunities to experience computer science. Integrated computing is often motivated as a way to introduce computing to students in a low-stakes environment, reducing barriers to learning computer science, often especially for underrepresented groups. This study explored integrated computing activities implemented in US schools to examine which programming and CT concepts they teach and whether those concepts differed across contexts. We gathered data on 262 integrated computing activities from in-service K-12 teachers and 20 contextual factors related to the classroom, the teacher, and the school. Our analysis revealed that programming and CT concepts were inversely related to five contextual factors, such that factors predicting more CT concepts also predicted fewer programming concepts. These factors reflected school resources, such that wealthier schools used activities with more CT concepts and fewer programming concepts. In addition, factors related to underrepresented groups often related to fewer concepts being taught in activities. School racial composition was the only factor of the 20 that had a relationship with both types of concepts in the same direction-students from underrepresented groups used activities with fewer CT and programming concepts. Our findings suggest that, although integrated computing can potentially introduce computing to a broader audience, we must evaluate what these activities are teaching and to whom. Another primary contribution of this work is an open-access dataset about integrated computing activities, the concepts that they teach, and the contextual factors around their use.

Self-efficacy is a reliable predictor of academic motivation and achievement across various disciplines and age groups, including computing education. Enhancing self-efficacy through instructional experiences and tools improves motivation and achievement, making it valuable in computing education. To make use of this relationship, educators and researchers must be able to accurately measure how experiences and tools affect self-efficacy. This study replicated and extended the validation of a new self-efficacy measurement for introductory programming students developed by Steinhorst et al. (2020). It replicated features of the original study, such as using other programming-specific self-efficacy measures. The study also introduced measures of general self-efficacy, collected data in new types of courses, collected data for a new programming language, and explored changes in self-efficacy throughout courses to assess validity. The results showed robust internal consistency and construct and convergent validity of the Steinhorst instrument for both introductory programming and data structures courses, aligning with general self-efficacy theory. The findings indicate the Steinhorst instrument's adaptability across different programming languages and contexts. A key insight is the need for researchers and educators to tailor the instrument by excluding items not yet covered in the curriculum to maintain its reliability. This research enhances the understanding of the Steinhorst instrument's robustness for assessing programming self-efficacy across diverse educational settings.

We explored how undergraduate introductory programming students naturalistically used generative AI to solve programming problems. We focused on the relationship between their use of AI to their self-regulation strategies, self-efficacy, and fear of failure in programming. In this repeated-measures, mixed-methods research, we examined students' patterns of using generative AI with qualitative student reflections and their self-regulation, self-efficacy, and fear of failure with quantitative instruments at multiple times throughout the semester. We also explored the relationships among these variables to learner characteristics, perceived usefulness of AI, and performance. Overall, our results suggest that student factors affect their baseline use of AI. In particular, students with higher self-efficacy, lower fear of failure, or higher prior grades tended to use AI less or later in the problem-solving process and rated it as less useful than others. Interestingly, we found no relationship between students' self-regulation strategies and their use of AI. Students who used AI less or later in problem-solving also had higher grades in the course, but this is most likely due to prior characteristics as our data do not suggest that this is a causal relationship.

Over the past decades, computer science education (CSEd) research has studied the multitude of factors that may impact student success in introductory programming courses (CS0/CS1). The lack of foundational structure behind how these factors interrelate has made it difficult to gain a thorough understanding of this area of CSEd literature. Gaining a deeper understanding and applying structure to these factors would allow CSEd to adopt better teaching practices, study habits, learning environments, course materials, etc. and to better understand the student experience to better foster success among a broader population of students. Our systematic literature review used search criteria for factors that predicted student success in CS0/CS1, which yielded 311 research articles. We then mapped this body of work under the Biggs' 3P (Presage, Process, Product) educational model, which provides a comprehensive framework for how students engage with learning opportunities. We discovered that although many studies focused on the Presage and Product phases of the model, fewer studies mapped to the Process phase, which describes the students' active learning processes. Our study shows there is a potential gap in the literature and future studies should focus more specifically on how students choose to engage with learning opportunities and what factors may be hindering that engagement throughout a learning period.

In our previous research we found that teaching novice programmers introductory programming in Java using subgoal labels led to deeper knowledge [2] and increased persistence for students potentially at risk of dropping out or failing their first undergraduate course in CS [3]. Subgoals are an instructional tool that is designed to bridge the gap between novices and experts, i.e., students and instructors. Experts often have difficulty explaining concepts at a level that novices understand because they have automatized much low-level knowledge. The task analysis used to identify subgoals makes this knowledge explicit. Subgoals are often expressed to students through subgoal-labeled worked examples that explicitly state the conceptual knowledge expressed through examples. This instructional design of examples allows students to see past superficial details of the example to the structural problem-solving procedure being exemplified [3].

Background and Context. Metacognitive skills are important for all students learning to program and interest in applying pedagogical approaches in early programming courses that focus on metacognitive aspects is growing. However, most studies of such approaches are not rigorously based in theory, and when they are, almost always utilize foundational education and psychology theories from as far back as the 1970s. More recent theory is less tested, and not all relevant metacognitive theories have been explored in the computing education research literature.

Across the globe, Computer Science Education has grown tremendously over the past decade to teach primary and secondary students computing ideas and tools. From integrating computational thinking in disciplines to teaching computer science as a stand alone subject, models for teacher preparation range from one and done professional learning workshops to full certificate and licensure programs. The group will focus on providing a landscape of how CS teachers are prepared academically in various countries and make evidence-based recommendations for how teachers should be educated to develop knowledge and skill to teach computer sci- ence. The working group will also discuss how to develop these knowledge systems while promoting instruction that is equitable and centers students in the classroom. In addition, the working group will focus on new directions in computing education (such as, artificial intelligence and machine learning) and their implica- tions for teacher preparation. We will bring together a group of international computer science education scholars who have been engaged in teacher preparation. In addition to what knowledge teachers need to teach CS, we will also focus on how the field is preparing teachers to think critically about AI/ML and the role of computer science in the design of technology tools to achieve goals while mitigating potential societal harms.

Metacognition and self-regulation are important skills for successful learning and have been discussed and researched extensively in the general education literature for several decades. More recently, there has been growing interest in understanding how metacognitive and self-regulatory skills contribute to student success in the context of computing education. This article presents a thorough systematic review of metacognition and self-regulation work in the context of computer programming and an in-depth discussion of the theories that have been leveraged in some way. We also discuss several prominent metacognitive and self-regulation theories from the literature outside of computing education—for example, from psychology and education—that have yet to be applied in the context of programming education. In our investigation, we built a comprehensive corpus of papers on metacognition and self-regulation in programming education, and then employed backward snowballing to provide a deeper examination of foundational theories from outside computing education, some of which have been explored in programming education, and others that have yet to be but hold much promise. In addition, we make new observations about the way these theories are used by the computing education community, and present recommendations on how metacognition and self-regulation can help inform programming education in the future. In particular, we discuss exemplars of studies that have used existing theories to support their design and discussion of results as well as studies that have proposed their own metacognitive theories in the context of programming education. Readers will also find the article a useful resource for helping students in programming courses develop effective strategies for metacognition and self-regulation.

For many decades, educational communities, including computing education, have debated the value of telling students what they need to know (i.e., direct instruction) compared to guiding them to construct knowledge themselves (i.e., constructivism). Comparisons of these two instructional approaches have inconsistent results. Direct instruction can be more efficient for short-term performance but worse for retention and transfer. Constructivism can produce better retention and transfer, but this outcome is unreliable. To contribute to this debate, we propose a new theory to better explain these research results. Our theory, multiple conceptions theory, states that learners develop better conceptual knowledge when they are guided to compare multiple conceptions of a concept during instruction. To examine the validity of this theory, we used this lens to evaluate the literature for eight instructional techniques that guide learners to compare multiple conceptions, four from direct instruction (i.e., test-enhanced learning, erroneous examples, analogical reasoning, and refutation texts) and four from constructivism (i.e., productive failure, ambitious pedagogy, problem-based learning, and inquiry learning). We specifically searched for variations in the techniques that made them more or less successful, the mechanisms responsible, and how those mechanisms promote conceptual knowledge, which is critical for retention and transfer. To make the paper directly applicable to education, we propose instructional design principles based on the mechanisms that we identified. Moreover, we illustrate the theory by examining instructional techniques commonly used in computing education that compare multiple conceptions. Finally, we propose ways in which this theory can advance our instruction in computing and how computing education researchers can advance this general education theory.

Metacognition and self-regulation are popular areas of interest in programming education, and they have been extensively researched outside of computing. While computing education researchers should draw upon this prior work, programming education is unique enough that we should explore the extent to which prior work applies to our context. The goal of this systematic review is to support research on metacognition and self-regulation in programming education by synthesizing relevant theories, measurements, and prior work on these topics. By reviewing papers that mention metacognition or self-regulation in the context of programming, we aim to provide a benchmark of our current progress towards understanding these topics and recommendations for future research. In our results, we discuss eight common theories that are widely used outside of computing education research, half of which are commonly used in computing education research. We also highlight 11 theories on related constructs (e.g., self-efficacy) that have been used successfully to understand programming education. Towards measuring metacognition and self-regulation in learners, we discuss seven instruments and protocols that have been used and highlight their strengths and weaknesses. To benchmark the current state of research, we examined papers that primarily studied metacognition and self-regulation in programming education and synthesize the reported interventions used and results from that research. While the primary intended contribution of this paper is to support research, readers will also learn about developing and supporting metacognition and self-regulation of students in programming courses.

The 32 chapters of the 2019 Cambridge Handbook of Computing Education Research synthesize the existing research in computing education and propose new directions for future research. An author from each chapter will summarize their chapter with auto-advancing slides. Attendees will be introduced to the breadth of content in the new handbook and can identify chapters of interest. This fits uniquely as a special session, and will likely be informative, inspiring, and overwhelming.

The subgoal learning framework has improved performance for novice programmers in higher education, but it has only started to be applied and studied in K-12 (primary/secondary). Programming education in K-12 is growing, and many international initiatives are attempting to increase participation, including curricular initiatives like Computer Science Principles and non-profit organizations like Code.org. Given that subgoal learning is designed to help students with no prior knowledge, we designed and implemented subgoals in the introduction to programming unit in Code.org's Computer Science Principles course. The redesigned unit includes subgoal-oriented instruction and subgoal-themed pre-written comments that students could add to their programming activities. To evaluate efficacy, we compared behaviors and performance of students who received the redesigned subgoal unit to those receiving the original unit. We found that students who learned with subgoals performed better on problem-solving questions but not knowledge-based questions and wrote more in open-ended response questions, including a practice Performance Task for the AP exam. Moreover, at least one-third of subgoal students continued to use the subgoal comments after the subgoal-oriented instruction had been faded, suggesting that they found them useful. Survey data from the teachers suggested that students who struggled with the concepts found the subgoals most useful. Implications for future designs are discussed.

This work extends previous research on subgoal labeled instructions by examining their effect across a semester-long, Java-based CS1 course. Across four quizzes, students were asked to explain in plain English the process that they would use to solve a programming problem. In this mixed methods study, we used the SOLO taxonomy to categorize student responses about problem-solving processes and compare students who learned with subgoal labels to those who did not. The use of the SOLO taxonomy classification allows us to look deeper than the mere correctness of answers to focus on the quality of the answers produced in terms of completeness of relevant concepts and explanation of relationships among concepts. Students who learned with subgoals produced higher-rated answers in terms of complexity and quality on three of four quizzes. Also, they were three times more likely to discuss issues of data type on a question about assignments and expressions than students who did not learn with subgoal labeling. This suggests that the use of subgoal labeling enabled students to gain a deeper and more complex understanding of the material presented in the course.

Learners' spatial skill is a reliable and significant predictor of achievement in STEM, including computing, education. Spatial skill is also malleable, meaning it can be improved through training. Most cognitive skill training improves performance on only a narrow set of similar tasks, but researchers have found ample evidence that spatial training can broadly improve STEM achievement. We do not yet know the cognitive mechanisms that make spatial skill training broadly transferable when other cognitive training is not, but understanding these mechanisms is important for developing training and instruction that consistently benefits learners, especially those starting with low spatial skill. This paper proposes the spatial encoding strategy (SpES) theory to explain the cognitive mechanisms connecting spatial skill and STEM achievement. To motivate SpES theory, the paper reviews research from STEM education, learning sciences, and psychology. SpES theory provides compelling post hoc explanations for the findings from this literature and aligns with neuroscience models about the functions of brain structures. The paper concludes with a plan for testing the theory's validity and using it to inform future research and instruction. The paper focuses on implications for computing education, but the transferability of spatial skill to STEM performance makes the proposed theory relevant to many education communities.

Subgoal learning has improved student problem-solving performance in programming, but it has been tested for only one-to-two hours of instruction at a time. Our work pioneers implementing subgoal learning throughout an entire introductory programming course. In this paper we discuss the protocol that we used to identify subgoals for core programming procedures, present the subgoal labels created for the course, and outline the subgoal-labeled instructional materials that were designed for a Java-based course. To examine the effect of subgoal labeled materials on student performance in the course, we compared quiz and exam grades between students who learned using subgoal labels and those who learned using conventional materials. Initial results indicate that learning with subgoals improves performance on early applications of concepts. Moreover, variance in performance was lower and persistence in the course was higher for students who learned with subgoals compared to those who learned with conventional materials, suggesting that learning with subgoal labels may uniquely benefit students who would normally receive low grades or dropout of the course.

As we celebrate the 50th SIGCSE Symposium, this panel explores how computing education researchers chart a course individually and as a community to build our research practices and collective knowledge of computing education. This navigation involves developing our research goals, which tools we use to work towards those goals, and which academic communities outside of computing education we seek to learn from and contribute to. However, these processes of navigation are rarely discussed as a community. Paper and grant submissions and reviews provide an imperfect way for our community to communicate our varied values and priorities. This panel brings together experts in computing education research who differ in their research goals, tools, and external communities. We can expect a lively discussion amongst the panelist and we hope to spark important discussions within the computing education research community!

Subgoal labeling is an instructional design framework for breaking down problems into pieces that are small enough for novices to grasp, and often difficult for instructors (i.e., experts) to articulate. Subgoal labels have been shown to improve student performance during problem solving in disciplines both in and out of computing. Improved student performance occurs because subgoal labels improve student transfer and retention of knowledge. With support from NSF (DUE-1712025, #1712231), subgoal labels have been identified and integrated into a CS1 course (variables, expressions, conditionals, loops, arrays, classes). This workshop will introduce participants to the materials and demonstrate how the subgoal labels and worked examples are integrated throughout the course. Materials include over 100 worked examples and practice problem pairs that increase in complexity and difficulty within each topic. The materials are designed to be integrated into CS1 courses as homework or classroom examples and activities. Assessment of topics using subgoal labels will also be discussed. Participants will also engage in an activity where they create an example for their own course using subgoal labels.

This Research Full Paper will discuss the demonstration of critical thinking of undergraduate students through their required co-op experiences. Critical thinking is an important 21st century skill that is expressed through measurable outcomes such as the ability to successfully use information, design, analyze, and problem solve. At Rochester Institute of Technology, a primarily technically focused university with strong programs in engineering and computing sciences, there is a dedicated effort to enhance the critical thinking ability of the students on campus. This paper will discuss our initiative for the integration of critical thinking into the student experience. In addition, the majority of our students complete one or more paid cooperative education experiences to satisfy their degree requirements. Upon completion of the experience, the employers are asked to rate the student performance on several metrics, one of which is demonstration of critical thinking. By analyzing the data from these surveys, we have found that employers consistently rate our students as proficient and capable. Our analysis shows that there is a slight difference in critical thinking ratings based on field of study, but not based on term in which it occurred, or number of experiences students engaged in.

Socioeconomic status (SES) has a measurable impact on many educational outcomes and likely also influences computer science (CS) achievement. We present a novel model to account for the observed connections between SES and CS achievement. We examined possible mediating variables between SES and CS achievement, including spatial ability and access to computing. We define access as comprised of measurements of prior learning opportunities for computing, perceptions of computer science, and encouragement to pursue computing. The factors (SES, spatial ability, access to computing, and CS achievement) were measured through surveys completed by 163 students in introductory computing courses at a college level. Through the use of exploratory structural equation modeling, we found that these variables do impact each other, though not as we originally hypothesized. For our sample of students, we found spatial ability was a mediating variable for SES and CS achievement, but access to computing was not. Neither model explained all the variance, and our subject pool of US college students had higher than average SES. Our findings suggest that SES does influence success in computer science, but that relationship may not be due to access to computing education opportunities. Rather, SES might be influencing variables such as spatial ability which in turn influence CS performance.

Our goal for the present research was to improve upon the subgoal learning framework and further enhance problem solving performance for novice programmers learning to use a block-based programming language. In particular, we are expanding upon recent work done by Margulieux and Morrison that prompts learners to self-explain the subgoals, or functional pieces, of a problem solving process to create their own instructional explanations of the process. We added to this work by exploring whether learners' self-explained instructions could be used to effectively scaffold initial problem solving attempts (i.e., practice problems) to further improve performance. In this experiment, learners self-explained subgoals using the most successful conditions from Margulieux and Catrambone's [1] prior work and then were given practice problems that were either unscaffolded (control condition), scaffolded with their own subgoal explanations, or scaffolded with explanations constructed by an instructional designer and computer scientist. Learners who were scaffolded with their own explanations performed better on later problem solving (i.e., an assessment test) than those scaffolded with the experts' explanations or those with no scaffolding. The results show that scaffolding initial problem solving with learners' explanations of the problem solving process can lead to better problem solving performance than scaffolding from experts if the learners construct explanations with adequate support.

A recent study about the effectiveness of subgoal labeling in an introductory computer science programming course both supported previous research and produced some puzzling results. In this study, we replicate the experiment with a different student population to determine if the results are repeatable. We also gave the experimental task to students in a follow-on course to explore if they had indeed mastered the programming concept. We found that the previous puzzling results were repeated. In addition, for the novice programmers, we found a statistically significant difference in performance based on whether the student had previous programming courses in high school. However, this performance difference disappears in a follow-on course after all students have taken an introductory computer science programming course. The results of this study have implications for how quickly students are evaluated for mastery of knowledge and how we group students in introductory programming courses.

We report on a study that used subgoal labels to teach students how to write while loops with a Parsons problem learning assessment. Subgoal labels were used to aid learning of programming while not overloading students' cognitive abilities. We wanted to compare giving learners subgoal labels versus asking learners to generate subgoal labels. As an assessment for learning we asked students to solve a Parsons problem -- to place code segments in the correct order. We found that students who were given subgoal labels performed statistically better than the groups that did not receive subgoal labels or were asked to generate subgoal labels. We conclude that a low cognitive load assessment, Parsons problems, can be more sensitive to student learning gains than traditional code generation problems.

Recent empirical results suggest that the instructional material used to teach computing may actually overload students' cognitive abilities. Better designed materials may enhance learning by reducing unnecessary load. Subgoal labels have been shown to be effective at reducing the cognitive load during problem solving in both mathematics and science. Until now, subgoal labels have been given to students to learn passively. We report on a study to determine if giving learners subgoal labels is more or less effective than asking learners to generate subgoal labels within an introductory CS programming task. The answers are mixed and depend on other features of the instructional materials. We found that student performance gains did not replicate as expected in the introductory CS task for those who were given subgoal labels. Computer science may require different kinds of problem-solving or may generate different cognitive demands than mathematics or science.

Computer-based learning environments can provide valuable resources for learning at scale, but students in these environments might learn without an instructor. Subgoal labels have been used in worked examples in STEM domains to help a learner understand the purpose of a set of steps, and this feature has increased problem solving performance [1]. Subgoal labels, however, have not been tested in instructional text. The present study explored this intervention. The results of the present study show that learners who received subgoal labels in both the text and example outperformed those in other conditions. When subgoal labeled text is paired with an unlabeled example, however, performance does not improve. These findings indicate that subgoal labeled instructional text when paired with subgoal labeled examples can improve performance in a computer-based learning environment.

Mental models are mental representations of how an action changes a problem state. Creating a mental model early in the learning process is a strong predictor of success in computer science classes. One major problem in computer science education, however, is that novices have difficulty creating mental models perhaps because of the cognitive overload caused by traditional teaching methods. The present study employed subgoal-labeled instructional materials to promote the creation of mental models when teaching novices to program in Android App Inventor. Utilizing this and other well-established educational tools, such as scaffolding, to reduce cognitive load in computer science education improved the performance of participants on novel tasks when learning to develop mobile applications.