Spatial skills have been shown to correspond with performance in computer science courses, and individuals further along in their computing journey tend to score higher on spatial skills tests. There is also evidence that these skills can be trained, consequently mitigating gaps in computing performance across demographic groups. However, it is unclear if training spatial skills has a lasting effect throughout a student's computing degree program. To fill this gap, we present longitudinal analyses from two separate institutions that implemented spatial skills training in their entry-level computing course. We perform a survival analysis on the retention data after four to five years and demonstrate that students with higher spatial skills after training are retained at higher rates throughout a degree program than students with lower spatial skills, even when the students initially started with lower spatial skills. However, the patterns of retention of students with high initial spatial skills and those who improve their spatial skills after training differ in our two cases, indicating the potential impact of other factors, such as self-regulation, which are critical for academic success. These patterns underscore that spatial-skill gains, while important, are not a panacea for student success in computing. Complementary supports that bolster self-regulation and related competencies may be required to sustain long-term retention.

In this poster, we present an ongoing project to develop and validate a subscale-based introductory computer science (CS1) assessment for broad use in computer science education research. This project addresses the need for a validated, adaptable assessment of CS1 knowledge across diverse institutional contexts and programming languages. We build upon the existing Second CS1 (SCS1) assessment, a pseudocode-based assessment for undergraduate introductory computer science course. While SCS1 has been widely used, it lacks flexibility and subscale validation, limiting its suitability for varied course content and research needs. Our poster will present our initial work towards creating an enhanced version of SCS1, termed SCS1++. We report on our progress developing and piloting a broader range of questions with varying difficulty levels. We will also use the poster as an opportunity to recruit field test sites for testing and validating SCS1++ with a broad student population.

Assessments of student learning can take many forms, but for assessing learning at scale, a multiple-choice exam is often used. A multiple-choice question is comprised of a stem, followed by a number of distractors and a correct option. All parts of a question need to be carefully designed to support assessment validity, but each part can also exhibit bias and negatively impact students in an inequitable way. In this research, we extend psychometric methods to explore the bias within the distractors of multiple choice questions on an introductory computing assessment for undergraduate students. We use Differential Distractor Functioning (DDF) on 259 student responses to identify problematic distractors. We discuss the distractors that were flagged in our analysis, which did vary in regards to biasing for male or female students. This work contributes a deeper understanding of the issues that can lie within an assessment, furthering our efforts to create fairer measures of learning for all students.

In a field as interdisciplinary as Computer Science Education, navigating the landscape of research paradigms and methodologies can be both exciting and overwhelming. Research design decisions are often constrained by time, disciplinary familiarity, and uncertainty about the benefits of alternative approaches. In Computer Science Education Research—an inherently interdisciplinary field—these constraints can prevent researchers from exploring the full range of possibilities available to them. Cards for Alternative Research Design (CARD) is designed to expand researchers' awareness of different research approaches through a card-based research design activity. By engaging with paradigms, methodologies, and constraints in a playful format, participants can critically reflect on their current research without the formality or potential confrontation of direct critique. This exercise strengthens the reasoning behind chosen approaches, while also surfacing valid arguments for adopting alternatives, all with a relatively low time investment.

Students' spatial skills are correlated with performance in STEM disciplines, including computer science. Fortunately, these skills can be trained, consequently improving computing course outcomes. The typical spatial skills training in computing education utilizes a workbook that helps students visualize and sketch the rotation of 3-D shapes. However, more methods have been tested in other STEM fields, including video games and spatial-related toys. In this position paper, we discuss these methods from adjacent fields and present a case of how to pilot them with our computer science students. We argue for the need for more flexible, but still impactful, spatial skills training methods to support our students. These methods could also provide a lower barrier to entry to teaching spatial skills, leading to more computing instructors adopting these methods and more computing students having spatial skills training opportunities. Further, broadening our spatial skills training methods can help us better understand the connections between spatial skills and computer science, which is especially obscure given the abstract nature of our field.

The application of mastery learning, where students progress through their learning in a self-paced manner until they have mastered specific concepts, is considered appealing for teaching introductory programming courses. Despite its growing popularity in computing and its extensive use in other disciplines, there is no overview of the design of courses that use mastery learning. In this position paper, we present an overview of five mastery learning models and discuss examples of how these can be applied in practice, both in foundational programming as well as more advanced courses. Our analysis focuses on the student progression through the course, the assessment structure, and the support for self-paced learning, including for struggling students. This work provides a greater understanding of mastery learning and its application in a computing education context.

Assessments that can measure student understanding of concepts in a reliable and valid way are incredibly valuable in research. Unfortunately, assessments can be a source of bias, differentially impacting students along various demographic lines. Differential Item Functioning (DIF) is a method to explore assessment bias. However, DIF is primarily limited to a single binary demographic variable (e.g. white and non-white; male and female). In this paper, we describe a novel expansion of DIF methods to explore intersections of student identities. We demonstrate the use of classic DIF on a data set of 255 complete responses to a CS1 assessment using binary race and gender variables in our analyses. Then, we present the importance of intersectional DIF by running a similar analysis on intersectional data. Using these methods, we identify problematic items on the assessment that are biased against certain groups of test-takers. Our work contributes an innovative method to help interpret assessment results and inform changes to assessments.

At ITiCSE 2021, Working Group 3 examined the evidence for teaching practices that broaden participation for women in computing, based on the National Center for Women & Information Technology (NCWIT) Engagement Practices framework. One of the report's recommendations was "Make connections from computing to your students' lives and interests (Make it Matter) but don't assume you know what those interests are; find out! " The goal of this 2023 working group is to find out what interests women students by bringing together data from our institutions on undergraduate module enrollment, seeing how they differ for women and men, and what drives those choices. We will code published module content based on ACM curriculum guidelines and combine these data to build a hierarchical statistical model of factors affecting student choice. This model should be able to tell us how interesting or valuable different topics are to women, and to what extent topic affects choice of module - as opposed to other factors such as the instructor, the timetable, or the mode of assessment. Equipped with this knowledge we can advise departments how to focus curriculum development on areas that are of value to women, and hence work towards making the discipline more inclusive.

The computing education (CEd) research community's growth is fueled in part by the growing number of CEd Ph.D. graduates, who are also increasingly entering postdoctoral positions and fellowships. While prior work has shown that postdoctoral researchers have positive effects on research labs, provide essential support to graduate students, and serve critical roles in the research ecosystem, there are no existing support structures for postdoctoral researchers within CEd. As postdocs navigate the challenges of their new positions alongside the demands of research and job searches, they would benefit from a community of peers wherein they could share their experiences and learn from others. This Birds-of-a-Feather (BoF) session is an organized space for CEd postdoctoral researchers, including those interested in postdoctoral positions, to build community and share postdoc experiences. Building on prior experience with running a similar BoF space for postdocs in SIGCSE 2022, the BoF discussion leaders is composed of current CEd postdocs, previous CEd postdocs who have taken on academic and research careers, and a CEd researcher from a non-academic space. This combination of diverse experiences, career contexts, and expertise will provide valuable perspectives that will aid in discussions of various postdoc experiences and research and career opportunities. BoF participants will have opportunities to discuss research goals, areas, and activities; exchange advice on navigating career paths and job searches; discuss approaches for mentorship within and outside research labs; and share best practices on the postdoc experience.

As computing education begins to grow in K-12 classrooms, it is increasingly important to understand the relationships between a student's coding attitudes and computing outcomes. While work has been done at the undergraduate and high school level, less is known about elementary students, and even less about Latinx students. We sought to explore the connections between Latinx elementary students' coding attitudes and computational thinking (CT) skills after participating in a year-long, remote computing curriculum in a public school district in Southern California. We collected validated measures on coding attitudes and CT skills for 164 fourth-grade students. We used structural equation modeling to analyze the relationship between the coding attitude constructs (i.e., confidence, utility, perception of coders, social influences, and interest) and scores on a CT assessment while controlling for demographic factors (i.e., gender, English language fluency, free or reduced lunch status, reading and math standardized test scores). In a model demonstrating acceptable fit, only math scores were a significant predictor of CT scores and coding confidence. Confidence was found to be significantly associated with the other attitude constructs except for perception of coders. Our results indicate that confidence plays a prominent role in student computing learning, especially at a young age. Efforts to better support Latinx students in computer science should include strategies that address students' coding confidence, such as developing positive perceptions of a student's learning process (e.g., failures as learning opportunities) in specific coding skills and practices (e.g., tinkering, debugging).

This study analyzed the relationship between computational thinking (CT) and coding attitudes of upper elementary students after exposure to a year-long CT curriculum. Using ordinary least squares regression and controlling for student demographics (i.e., gender and English Learner (EL) status), we found that CT skills were a significant factor in modeling coding attitudes, regardless of demographic controls. Interviews of the students unveiled an interaction between coding interest and social values in a debugging process.

In the age of computing, there still exist many schools that do not offer computer science courses. The reason can be esoteric to designers of interventions, curricula, and policies. This study aims to answer the research question: What do school officials perceive as barriers to and supports for offering computer science courses at their school? This article provides a case study of four public high schools in the state of Georgia to better understand the barriers to adopting computer science. Interviews conducted with school principals, counselors, and teachers were thematically analyzed to study each case. The pattern that emerged from these themes is framed by structuration theory, wherein the structures (school policies, resources, and courses) influence the agents (teachers, students, the community), and in turn, the agents influence the structures. This framing provides insights into what attributes can be supports for or barriers against schools offering computer science courses.

Background and Context. With increasing efforts to bring computing education opportunities into elementary schools, there is a growing need for assessments, with arguments for validity, to support research evaluation at these grade levels. After successfully piloting a 10-question computational thinking assessment (Assessment of Computing for Elementary Students – ACES) for 4th graders in Spring 2020, we used our analyses of item difficulty and discrimination to iterate on the assessment. Objectives. To increase the number of potential items for ACES, we created isomorphic versions of existing questions. The nature of the changes varied from incidental changes that we did not believe would impact student performance to more radical changes that seemed likely to influence question difficulty. We sought to understand the impact of these changes on student performance. Method. Using these isomorphic questions, we created two versions of our assessment and piloted them in Spring 2021 with 235 upper-elementary (4th grade) students. We analyzed the reliability of the assessments using Cronbach’s alpha. We used Chi-squared tests to analyze questions that were identical across the two assessments to form a baseline of comparison and then ran Chi-Squared and Kruskal-Wallis H tests to analyze the differences between the isomorphic copies of the questions. Findings. Both assessment versions demonstrated good reliability, with identical Cronbach’s alphas of 0.868. We found statistically similar performance on the identical questions between our two groups of students, allowing us to compare their performance on the isomorphic questions. Students performed differently on the isomorphic questions, indicating the changes to the questions had a differential impact on student performance. Implications. This paper builds on existing work by presenting methods for creating isomorphic questions. We provide valuable lessons learned, both on those methods and on the impact of specific types of changes on student performance.

There is a growing number of Ph.D. graduates whose research focuses on computing education, and they are significantly fueling the growth of the computing education research community. As more computing education Ph.D. graduates weigh their options on the job market, they are increasingly entering postdoctoral positions. For example, seven Computing Innovation Fellows positions over the last two years have been awarded to computing education researchers. Research shows that postdoctoral researchers have positive, cascading effects on research labs. However, despite the critical role they play in the research ecosystem, and the growing importance of postdoctoral positions in career development, there are no support structures for postdoctoral researchers in the computing education research community. This Birds-of-a-Feather session is an organized opportunity for postdoctoral researchers and those interested in postdoctoral positions to connect with one another, share postdoc experiences, and generate best practices. Attendees will have opportunities to discuss research goals and activities, career trajectories and opportunities, future conference and publication plans, pathways for future collaborations, and advice for the postdoc experience.

As computing education expands to ''all'' students, so too must the assessment of computational learning. However, there are many challenges to designing and using computing assessments in a valid and reliable way. This is especially true with respect to pre-tests, or assessments given at the beginning of an intervention, course, or study. For elementary and middle school interventions, it is still likely that many students in any given study sample will have had no prior experience with computing. For high school interventions, students may have a wide range of prior experiences. How do you design or select a pre-test for these situations? In this poster, we discuss the design of pre-tests for the computing education research community. We outline the fundamental principles of pre-tests and the different purposes they serve in research studies. We complement these principles with examples of pre-tests used in current computing education research. This poster aims to provide guidance on how to intentionally develop and use pre-tests to strengthen the validity of our research findings and better inform on student learning outcomes.

Validated instruments of knowledge are important for creating an accepted and shared measurement for the most important part of education – student learning. The Force Concept of Inventory from Physics Education Research has had a significant impact across STEM education. The Foundational CS1 (FCS1) and Second CS1 (SCS1) assessments were created and validated to further computing education research. Now, ten years after the publication of the FCS1 assessment and five years after the release of the SCS1 assessment, we can trace the use and the impact that these validated instruments have had on the needs and knowledge of the computing education community. In this paper, we examine how the FCS1 and SCS1 assessments have been used and adapted. We use this discussion to guide a comparison between our field and physics education to give a sense of direction to future research. In looking back on the use of these validated instruments, we can better understand our needs in computing education research and about our future needs in assessment.

Computing has, for many years, been one of the least demographically diverse STEM fields, particularly in terms of women's participation [2] and those from minoritized racial and ethnic groups. In the case of higher education, one of the most powerful sites of intervention is the classroom. The last decade has seen a proliferation of research exploring new teaching techniques and course sequencing and their effect on the retention of students who have historically been excluded from computing. This research suggests interventions and practices that can affect the inclusiveness of the computer science classroom and potentially improve learning outcomes for all students. But research needs to be translated into practice, and practices need to be taken up in real classrooms. The goal of this working group (WG) is to conduct a systemic "state-of-the-art" review of recent empirical studies of teaching practices that have some explicit test of the impact on women (or other under-represented groups) in computing. The WG will produce an annotated bibliography and a report that distills the research into specific, actionable practices.

As reliance on technology increases in practically every aspect of life, all students deserve the opportunity to learn to think computationally from early in their educational experience. To support the kinds of computer science curriculum and instruction that makes this possible, there is an urgent need to develop and validate computational thinking (CT) assessments for elementary-aged students. We developed the Assessment of Computing for Elementary Students (ACES) to measure the CT concepts of loops and sequences for students in grades 3-5. The ACES includes block-based coding questions as well as non-programming, Bebras-style questions. We conducted cognitive interviews to understand student perspectives while taking the ACES. We piloted the assessment with 57 4th grade students who had completed a CT curriculum. Preliminary analyses indicate acceptable reliability and appropriate difficulty and discrimination among assessment items. The significance of this paper is to present a new CT measure for upper elementary students and to share its intentional development process.

Increased opportunities for online learning, including growth in Massive Open Online Courses (MOOCS), are changing our education environments, increasing access and flexibility in how students engage with education. However, there are still many questions regarding how we engage with students effectively in these environments, in particular through assessment.

An estimated 35% of high school principals across the U.S. report teaching computer science (CS) at their schools, according to a 2018 code.org access report. Meanwhile, a growing number of organizations have missions of providing computer science to all students in primary and secondary schools. In order to reach all students with CS, we need to understand the choices students, teachers, and schools make regarding accessing CS education. In this poster, we present a regression model that predicts the percentage of students enrolled in a computer science course at a public high school in the U.S. state of Georgia.We choose a single state because the US school system is highly-distributed, so policy and standards contexts differ between states. Using publicly available data sets, we explore what could advance or diminish a school's choice to offer a CS course. Our results show that the explanatory variable that mostly highly correlates with and most strongly influences CS being offered in a school is whether CS has been offered before. The implication is that startup costs and inertia may be among the most critical factors, as opposed to wealth or size of the school district. Median income at the county level and enrollment numbers at a school do affect CS enrollment numbers, but only explain a small amount of the variance. We are currently undertaking qualitative work to understand the idiosyncratic factors that influence schools' decision to offer a CS course.

A meritocracy is an environment in which rewards are based solely on ability. The concept of privilege runs counter to this idea. Privilege is an unearned, unasked for advantage gained because of the way society views an aspect of your identity such as race, ethnicity, gender, perceived socioeconomic status, and language. Privilege can affect our students experiences in the classroom and beyond. However, privilege does not have to be viewed as an entitlement, but rather an opportunity to level the playing field for all people. In this session, we seek to visually display the world of privilege through adapting interactive activities known as a privilege walk and an asset walk. Come to learn and meet with other attendees who want to create a more inclusive society

Socioeconomic status (SES) has a measurable impact on many educational outcomes and likely also influences computer science (CS) achievement. We present a novel model to account for the observed connections between SES and CS achievement. We examined possible mediating variables between SES and CS achievement, including spatial ability and access to computing. We define access as comprised of measurements of prior learning opportunities for computing, perceptions of computer science, and encouragement to pursue computing. The factors (SES, spatial ability, access to computing, and CS achievement) were measured through surveys completed by 163 students in introductory computing courses at a college level. Through the use of exploratory structural equation modeling, we found that these variables do impact each other, though not as we originally hypothesized. For our sample of students, we found spatial ability was a mediating variable for SES and CS achievement, but access to computing was not. Neither model explained all the variance, and our subject pool of US college students had higher than average SES. Our findings suggest that SES does influence success in computer science, but that relationship may not be due to access to computing education opportunities. Rather, SES might be influencing variables such as spatial ability which in turn influence CS performance.

Online education is an important tool for supporting the growing number of teachers and students in computer science. We created two eBooks containing interactive content for Advanced Placement Computer Science Principles, one targeted at teachers and one at students. By comparing the eBook usage patterns of these populations, including activity usage counts, transitions between activities, and pathways through the eBook, we develop a characterization of how student use of the eBook differs from teacher use. We offer design recommendations for how eBooks might be developed to target each of our populations. We ground our recommendations in a theory of teachers as expert learners who possess a greater ability to regulate their own learning process.

Computer science is rapidly expanding across the United States, and as schools look for guidance about what constitutes developmentally appropriate topics, state education departments are looking for assistance in the creation of standards. The K--12 Computer Science (CS) Framework was developed to fit these needs. The framework is the first of its kind in the K--12 CS domain, filling a much-needed structural and definitional role in the field. The framework was created as part of a collaboration at multiple levels within the CS community across the United States. Following the footsteps of other disciplines, this framework provides definitions and guidelines on what students should know (concepts) and be able to do (practices) within certain grade bands in today's computing classrooms. This paper details why the framework was developed, how it was designed, and what impacts it could have on the future of K--12 computing education.

Several countries are trying to provide access to computing education for all secondary students. However, there are not enough teachers who are prepared to teach computer science. Interactive electronic books (ebooks) are a promising approach for providing low-cost professional development in computer science. Over the last four years, our research group has been conducting design-based research by iteratively developing and testing versions of a teacher ebook to help secondary teachers with no programming experience learn to teach an introductory programming course. The interactive elements in the ebook were designed based on research results from educational psychology and are intended to make learning more efficient and effective. Our goals for this effort are to increase teachers' knowledge of computer science concepts and to improve teachers' confidence in their ability to teach computer science. In this paper we summarize our previous work and report on a large-scale study of version two of the teacher ebook. We also recommend several design principles for interactive ebooks for computing teachers based on feedback from teachers, log file analyses, and randomized controlled studies.

Computing education lags other discipline-based education research in the number and range of validated assessments available to the research community. Validated assessments are important for researchers to reduce experimental error due to flawed assessments and to allow for comparisons between different experiments. Although the need is great, building assessments from scratch is difficult. Once an assessment is built, it's important to be able to replicate it, in order to address problems within it, or to extend it. We developed the Second CS1 Assessment (SCS1) as an isomorphic version of a previously validated language-independent assessment for introductory computer science, the FCS1. Replicating the FCS1 is important to enable free use by a broader research community. This paper is documentation of our process for replicating an existing validated assessment and validating the success of our replication. We present initial use of SCS1 by other research groups, to serve as examples of where it might be used in the future. SCS1 is useful for researchers, but care must be taken to avoid undermining the validity argument.

Female students are underrepresented in computing careers and in computing majors at the college or university level. This underrepresentation starts in high school. Only 20% of Advanced Placement (AP) Computer Science (CS) A exam takers in 2014 nationally were female. In addition to being underrepresented, female students tend to have a lower pass rate on the exam than male students. Failing the exam could reduce interest in computing as a career. Sisters Rise Up 4 CS was created at the Georgia Institute of Technology (Georgia Tech) to help more female high school students pass the AP CS A exam. It offers help sessions, near-peer role models, exposure to a college campus, and a community of learners. Sisters Rise Up 4 CS is based on Project Rise Up 4 CS, which was also created at Georgia Tech to help increase the number of African-American students that pass the AP CS A exam. However, Sisters Rise Up 4 CS does not include the financial incentives offered by Project Rise Up 4 CS. Project Rise Up 4 CS has resulted in statistically significant positive changes in attitudes for participants and a large increase in the participants' perception of their ability to pass the exam. Even students who did not pass the exam have reported positive benefits from being in Project Rise Up 4 CS. This paper summarizes the findings from Project Rise Up 4 CS from 2013-2015 and presents the findings from the pilot of Sisters Rise Up 4 CS from 2014-2015. We used a mixed-methods approach to evaluate the project and to judge the effectiveness of the project elements.

Validated assessments are important for teachers and researchers. A validated assessment is carefully developed to make sure that it is measuring the right things. Computing education needs more and better validated assessments. Validated assessments provide instructors with insight on how their students are doing in their class and provide researchers with insight on whether certain technologies and interventions are successful. Building high-quality, validated assessments is difficult. However, it is possible to replicate an existing validated assessment, and the new assessment can be validated against the original assessment. We need mechanisms to replicate assessments so that we can build more and more varied assessments for different audiences. We developed the Secondary CS1 Assessment (SCS1) as an isomorphic version of a previously validated assessment instrument for introductory computer science. In this poster we provide an overview for the process of replicating an existing valid knowledge assessment and validating the replication. Handouts will be provided with information about how to access and use the SCS1 Assessment.

My research investigates the interaction between privilege and computer science education, and asks what more can be done to level the playing field. Privilege is an unearned, unasked-for advantage gained because of the way society views an aspect of a student's identity, such as race, ethnicity, gender, socioeconomic status, and language. Privilege may provide advantages to some students, and under-privileged students may face unfair barriers to success in education.

Students who attend academic conferences can broaden their horizons, increase their commitment to the discipline, find role models and mentors, and gain concrete opportunities for work and study. However, attending one's first conference can be an overwhelming experience. We often focus on the procurement of funding to send students to conferences, and do not spend as much time considering what will happen once they get there. In this BoF, we will share strategies for guiding and supporting undergraduates to take full advantage of attending a CS conference.