Background and Context. There are growing efforts to increase the percentage of secondary students who take computing courses. However, U.S. women identifying as Black/African American, Hispanic/Latina, and/or Native American remain underrepresented in computing and face persistent challenges in learning to program. Moreover, it remains underexplored what types of programming learning support minoritized high school women desire.

Cybersecurity professionals must anticipate how attackers exploit system weaknesses -- a perspective known as ''adversarial thinking'' (AT). While cybersecurity education emphasizes the development of AT skills, there is little consensus on what these skills entail or how to assess them. We conducted a scoping review of computing education literature and semi-structured interviews (N=8) with cybersecurity professionals to better understand how adversarial thinking (AT) manifests in both academia and industry. Our analysis of 19 academic papers found that adversarial thinking is frequently described through system-centric approaches and/or hacker-centric definitions, with both presenting it as an open-ended creative exploration. However in practice, time constraints force cybersecurity professionals to develop a risk-based mindset and rely on institutionalized adversarial knowledge rather than a constant creative analysis. Our professionals strategically deploy adversarial thinking when standardized checklists feel inadequate or when potential risks warrant a deeper investigation. This paper contributes an empirically grounded account of how AT operates under professional constraints, and identify implications for how cybersecurity education can better prepare students.

Students learning programming exercise agency in deciding when and how to use GenAI tools like ChatGPT. However, this agency is often implicit and shaped by deadline pressure and peer behavior rather than explicit and conscious learning goals. We designed a GenAI Contract grounded in harm reduction and self-regulated learning theory to scaffold intentional decision-making: students articulated personal learning goals, created usage guidelines, and reflected on alignment at strategic points across an eleven-week semester. The contract was non-binding and graded only for completion, emphasizing self-awareness over enforcement. We implemented this with N=217 students in an intermediate Python course. For students still forming their relationship with GenAI, it worked, as 58% of students reported the intervention changing their thinking and created helpful accountability structures. However, awareness did not always translate to sustained behavior change. Some students who valued their guidelines still abandoned them under various pressures. Maintaining guidelines required constant self-control across hundreds of decisions, while using GenAI freely requires none. Many students could not sustain this burden despite this self-awareness. We discuss supporting student agency when GenAI tools and learning goals create tension.

Decades of research on Peer Instruction (PI) have shown that it improves student engagement, retention, and learning in many fields, including computing. In PI, as defined by Eric Mazur of Harvard, the instructor displays a difficult multiple-choice question during lecture which students first answer individually, then discuss with nearby peers, and then answer individually again. We created a free tool, Peer+, for Peer Instruction and tested it over several semesters at multiple institutions (University of Michigan, Duke University, Virginia Tech, and Berea College) on over 1,000 students. Peer+ is freely available in the open-source ebook platform Runestone Academy. Instructors can use any of the 90+ free ebooks on that platform or use an empty ebook. There are ebooks for Advanced Placement Computer Science, CS1 (Python, Java, and C++), CS2 in C++ and Java, and Discrete Math. We added hundreds of existing PI questions to these ebooks, which instructors can use or they can create their own multiple-choice questions. In this tutorial you will learn best practices for Peer Instruction and gain hands-on experience creating PI assignments, browsing existing PI questions, writing new PI questions, and grading PI assignments. You will also experience PI sessions using three different modes for peer discussion: traditional verbal discussion with nearby peers, text-chat with assigned group members, and text-chat with an LLM for students answering questions after lecture.

In Peer Instruction (PI) an instructor displays a challenging multiple-choice question during lecture that students answer individually, discuss verbally with nearby peers, answer individually again, and finally, the instructor leads a discussion of the question. Peer Instruction typically increases student learning and motivation over traditional lecture. We added a text-chat mode to improve PI for remote synchronous learning. This feature assigns students to discussion groups to maximize the number of groups that have members with different answers. The tool was pilot tested in Winter 2022 and revised. In Fall 2022 and Winter 2023, it was tested at one institution. In Fall 2024, it was tested at four institutions. We conducted a log file analysis of student data from 1394 students and analyzed surveys with 848 student responses. We found that questions answered using the text-chat had a significantly higher improvement than those using traditional verbal discussion, although the two modes were tested with different questions. Interestingly, most of the students preferred to discuss the question verbally, although some preferred the text-chat discussion. These results inform efforts to improve the effectiveness of Peer Instruction and increase its adoption.

Despite decades of research on the effectiveness of Peer Instruction (PI), it can be hard to convince computing instructors to try it. In Peer Instruction an instructor displays a hard multiple-choice question during lecture, students answer individually, discuss with peers, and answer independently again. The main reasons computing instructors give for not adopting PI is a lack of awareness, concerns that it would result in less coverage of material in lecture, and the time needed to adopt it. We tried to increase awareness of PI and address instructors' concerns during a three-day summer instructor workshop with 14 instructors in 2024. A pre-survey was administered on the first day of the workshop and a post-survey on the last day. We administered a second post-survey in the fall of 2024. Later semi-structured interviews were conducted with nine instructors. The adoption rate from the 2024 workshop was over 50%, which was much higher than the previous two workshop's adoption rate of 23%. The workshop also increased knowledge of PI and alleviated several concerns. However, instructors wanted more follow-up support, an easier and faster way to create new PI questions, and better integration with their Learning Management System (LMS). Instructors who adopted the free tool, Peer+, reported positive student reactions and better understanding of student misconceptions. All instructors who adopted Peer+ plan to continue to use it.

While generative AI has demonstrated potential for personalized learning, concerns have emerged about the risk of student overreliance and passive use, such as submitting the AI-generated code without even reading it. Parsons puzzles require students to select and arrange mixed-up code blocks into the correct sequence. They provide a simplified yet authentic problem-solving experience that offers targeted support for students struggling to program independently while fostering cognitive engagement. We developed a tool that generates personalized Parsons puzzles based on students' written code, designed to assist students who find it difficult to write short code solutions on their own. Such personalization can occur at both the solution & block levels, or at the solution level alone. This tool is embedded in a free ebook platform, Runestone Academy. Runestone has over 90 free ebooks for computing and math classes and was used by over 80,000 students in 2024-2025. The tool supports two main programming languages: Python and Java. In this workshop, we will guide instructors in using Runestone, creating personalized Parsons puzzles, and creating, reviewing, and grading assignments. Participants should bring laptops (with access to power and tables) and have their own valid API keys to access AI models.

As computing's societal impact grows, so does the need for computing students to recognize and address the ethical and sociotechnical implications of their work. While there are efforts to integrate ethics into computing curricula, we lack a standardized tool to measure those efforts, specifically, students' attitudes towards ethical reflection and their ability to effect change. This paper introduces the novel framework of Critically Conscious Computing and reports on the development and content validation of the Critical Reflection and Agency in Computing Index, a novel instrument designed to assess undergraduate computing students' attitudes towards practicing critically conscious computing. The resulting index is a theoretically grounded, expert-reviewed tool to support research and practice in computing ethics education. This enables researchers and educators to gain insights into students' perspectives, inform the design of targeted ethics interventions, and measure the effectiveness of computing ethics education initiatives.

Even though AI literacy has emerged as a prominent education topic in the wake of generative AI, its definition remains vague. There is little consensus among researchers and practitioners on how to discuss and design AI literacy interventions. The term has been used to describe both learning activities that train undergraduate students to use ChatGPT effectively and having kindergarten children interact with social robots. This paper applies an integrative review method to examine empirical and theoretical AI literacy studies published since 2020. In synthesizing the 124 reviewed studies, three ways to conceptualize literacy-functional, critical, and indirectly beneficial-and three perspectives on AI-technical detail, tool, and sociocultural-were identified, forming a framework that reflects the spectrum of how AI literacy is approached in practice. The framework highlights the need for more specialized terms within AI literacy discourse and indicates research gaps in certain AI literacy objectives.

Computing ethics education aims to develop students' critical reflection and agency. We need validated ways to measure whether our efforts succeed. Through two survey administrations (N=474, N=464) with computing students and professionals, we provide evidence for the validity of the Critical Reflection and Agency in Computing Index. Our psychometric analyses demonstrate distinct dimensions of ethical development and show strong reliability and construct validity. Participants who completed computing ethics courses showed higher scores in some dimensions of ethical reflection and agency, but they also exhibited stronger techno-solutionist beliefs, highlighting a challenge in current pedagogy. This validated instrument enables systematic measurement of how computing students develop critical consciousness, allowing educators to better understand how to prepare computing professionals to tackle ethical challenges in their work.

Despite evidence of its effectiveness, Peer Instruction (PI) has not been widely adopted by undergraduate computing instructors. In PI, an instructor displays a hard multiple-choice question that students answer individually, then discuss their answer with peers, then answer again, and finally an instructor leads a discussion of the question. Even though the benefits of PI are well documented, it can be difficult to convince computing instructors to move away from passive lectures. Major reasons why instructors do not adopt PI include a lack of awareness, lack of time, and concerns over their ability to cover content. We hypothesized that we could encourage the adoption of PI by creating Peer+, a free tool in an ebook platform, a searchable question bank, and running summer instructor workshops. We offered a three-day in-person summer workshop to a total of 37 instructors in 2022 and 2023. Instructors completed a pre-survey, immediate post-survey, and a follow-up post survey after the fall semester. We also conducted semi-structured interviews with 17 instructors. On the immediate post-survey most (33/37, 89%) instructors reported that they were very likely or likely to use the tool in the fall. However, on the follow-up survey, less than a quarter (6/26, 23%) actually did. The number one reason for not using the tool was a lack of time (18/26, 69%). Notably, all of the instructors who used Peer+ planned to use it again. This work informs efforts to increase the adoption of evidence-based pedagogical approaches in computing.

Equity and diversity issues persist in computer science education. Prior research has identified marginalizing factors prevalent in the computing fields that discourage women and people of color, while also highlighting experiences that help them persist. Extending prior studies and applying an intersectional lens, this study aims to center the voice of Black girl high school students and present their experiences taking Advanced Placement (AP) Computer Science (CS) courses. We investigate how and why they persisted by conducting semi-structured interviews with eight Black girl students who have completed at least one AP CS course, and connect their stories to prior work. Applying situated expectancy-value theory (SEVT), our findings reveal examples of marginalizing and empowering social interactions within learning environments, students' awareness and active construction of social support networks, and conscious effort to resist inequity as motivation for studying CS. Our findings build upon prior studies of Black women's experiences in post-secondary and computing careers, and extend this research to secondary education contexts.

As generative AI products could generate code and assist students with programming learning seamlessly, integrating AI into programming education contexts has driven much attention. However, one emerging concern is that students might get answers without learning from the LLM-generated content. In this work, we deployed the LLM-powered personalized Parsons puzzles as scaffolding to write-code practice in a Python learning classroom (PC condition) and conducted an 80-minute randomized between-subjects study. Both conditions received the same practice problems. The only difference was that when requesting help, the control condition showed students a complete solution (CC condition), simulating the most traditional LLM output. Results indicated that students who received personalized Parsons puzzles as scaffolding engaged in practicing significantly longer than those who received complete solutions when struggling.

As discussions of computing's impact on society increase in public discourse, so does recognition for computing students to address the ethical and sociotechnical implications of their work. While efforts to integrate issues of ethics and social justice into computing curricula are nascent, we lack a standardized measure to monitor our progress towards these goals. In this poster, we report on the development and validation of the Critical Reflection and Agency in Computing Index, a novel instrument designed to assess undergraduate computing students' attitudes towards practicing critically conscious computing. The resulting index is a theoretically grounded, expert-reviewed tool with evidence for reliability and validity to support research and practice in computing ethics education. This enables researchers and educators to gain insights into students' perspectives, inform the design of targeted ethics interventions, and monitor the effectiveness of computing ethics education initiatives.

This paper investigates using micro Parsons problems as a novel practice approach for learning Structured Query Language (SQL). In micro Parsons problems learners arrange predefined code fragments to form a SQL statement instead of typing the code. SQL is a standard language for working with relational databases. Targeting beginner-level SQL statements, we evaluated the efficacy of micro Parsons problems with block-based feedback and execution-based feedback compared to traditional text-entry problems. To delve into learners’ experiences and preferences for the three problem types, we conducted a within-subjects think-aloud study with 12 participants. We found that learners reported very different preferences. Factors they considered included perceived learning, task authenticity, and prior knowledge. Next, we conducted two between-subjects classroom studies to evaluate the effectiveness of micro Parsons problems with different feedback types versus text-entry problems for SQL practice. We found that learners who practiced by solving Parsons problems with block-based feedback had a significantly higher learning gain than those who practiced with traditional text-entry problems.

The capability of large language models (LLMs) to generate, debug, and explain code has sparked the interest of researchers and educators in undergraduate programming, with many anticipating their transformative potential in programming education. However, decisions about why and how to use LLMs in programming education may involve more than just the assessment of an LLM’s technical capabilities. Using the social shaping of technology theory as a guiding framework, our study explores how students’ social perceptions influence their own LLM usage. We then examine the correlation of self-reported LLM usage with students’ self-efficacy and midterm performances in an undergraduate programming course. Triangulating data from an anonymous end-of-course student survey (n = 158), a mid-course self-efficacy survey (n=158), student interviews (n = 10), self-reported LLM usage on homework, and midterm performances, we discovered that students’ use of LLMs was associated with their expectations for their future careers and their perceptions of peer usage. Additionally, early self-reported LLM usage in our context correlated with lower self-efficacy and lower midterm scores, while students’ perceived over-reliance on LLMs, rather than their usage itself, correlated with decreased self-efficacy later in the course.

Learning to program can be challenging, and providing high-quality and timely support at scale is hard. Generative AI and its products, like ChatGPT, can create a solution for most intro-level programming problems. However, students might use these tools to just generate code for them, resulting in reduced engagement and limited learning. In this paper, we present CodeTailor, a system that leverages a large language model (LLM) to provide personalized help to students while still encouraging cognitive engagement. CodeTailor provides a personalized Parsons puzzle to support struggling students. In a Parsons puzzle, students place mixed-up code blocks in the correct order to solve a problem. A technical evaluation with previous incorrect student code snippets demonstrated that CodeTailor could deliver high-quality (correct, personalized, and concise) Parsons puzzles based on their incorrect code. We conducted a within-subjects study with 18 novice programmers. Participants perceived CodeTailor as more engaging than just receiving an LLM-generated solution (the baseline condition). In addition, participants applied more supported elements from the scaffolded practice to the posttest when using CodeTailor than baseline. Overall, most participants preferred using CodeTailor versus just receiving the LLM-generated code for learning. Qualitative observations and interviews also provided evidence for the benefits of CodeTailor, including thinking more about solution construction, fostering continuity in learning, promoting reflection, and boosting confidence. We suggest future design ideas to facilitate active learning opportunities with generative AI techniques.

Peer Instruction (PI) is a learning activity that lets students solve a difficult multiple-choice question individually, submit their answer, discuss with peers to solve the problem collaboratively, and then submit the answer again. Despite plentiful evidence to support its effectiveness, PI has not been widely adopted by undergraduate computing instructors due to low awareness of PI, the effort needed to create PI questions, the limited instructional time needed for PI activities during lectures, and potential adverse reactions from students.

Novice programmers need to write basic code as part of the learning process, but they often face difficulties. To assist struggling students, we recently implemented personalized Parsons problems, which are code puzzles where students arrange blocks of code to solve them, as pop-up scaffolding. Students found them to be more engaging and preferred them for learning, instead of simply receiving the correct answer, such as the response they might get from generative AI tools like ChatGPT. However, a drawback of using Parsons problems as scaffolding is that students may be able to put the code blocks in the correct order without fully understanding the rationale of the correct solution. As a result, the learning benefits of scaffolding are compromised. Can we improve the understanding of personalized Parsons scaffolding by providing textual code explanations? In this poster, we propose a design that incorporates multiple levels of textual explanations for the Parsons problems. This design will be used for future technical evaluations and classroom experiments. These experiments will explore the effectiveness of adding textual explanations to Parsons problems to improve instructional benefits.

Amid increasing calls for critical and anti-oppressive approaches to computer science (CS) education, educators are exploring how to create justice-centered teaching material. Additionally, broadening participation in justice-centered computing requires an understanding of students' relationship with social justice and their CS education. In this study, we created and distributed a programming project with a social justice context and critical thinking reflection questions as a probe for an intermediate programming class. We conducted a thematic analysis of 11 semi-structured interviews and distributed a short survey (N=86) with students of this class at a large public research university in the American Midwest. Our findings showed that these students support social justice contexts and content within their computer science education. Students requested deeper dives and discussions into social justice programming that would challenge their preconceived notions, incorporate calls to action, and direct action. However, we also found an interesting tension forming: many students described how their homework problem-solving mindset clashed with the critical thinking reflection questions.

AI code generators like OpenAI Codex have the potential to assist novice programmers by generating code from natural language descriptions, however, over-reliance might negatively impact learning and retention. To explore the implications that AI code generators have on introductory programming, we conducted a controlled experiment with 69 novices (ages 10-17). Learners worked on 45 Python code-authoring tasks, for which half of the learners had access to Codex, each followed by a code-modification task. Our results show that using Codex significantly increased code-authoring performance (1.15x increased completion rate and 1.8x higher scores) while not decreasing performance on manual code-modification tasks. Additionally, learners with access to Codex during the training phase performed slightly better on the evaluation post-tests conducted one week later, although this difference did not reach statistical significance. Of interest, learners with higher Scratch pre-test scores performed significantly better on retention post-tests, if they had prior access to Codex.

Decades of research have provided evidence of the effectiveness of Peer Instruction (PI) in many fields, including computing. PI involves an instructor displaying a hard multiple-choice question that students answer individually, then discuss with peers and answer again. The instructor then displays the results from the two votes and leads a discussion.

Regular expressions (regex) are a text processing method widely used in data analysis, web scraping, and input validation. However, students find regular expressions difficult to create since they use a terse language of characters. Parsons problems can be a more efficient way to practice programming than typing the equivalent code with similar learning gains. In traditional Parsons problems, learners place mixed-up fragments with one or more lines in each fragment in order to solve a problem. To investigate learning regex with Parsons problems, we introduce micro Parsons problems, in which learners assemble fragments in a single line. We conducted both a think-aloud study and a large-scale between-subjects field study to evaluate this new approach. The think-aloud study provided insights into learners' perceptions of the advantages and disadvantages of solving micro Parsons problems versus traditional text-entry problems, student preferences, and revealed design considerations for micro Parsons problems. The between-subjects field study of 3,752 participants compared micro Parsons problems with text-entry problems as an optional assignment in a MOOC. The dropout rate for the micro Parsons condition was significantly lower than the text-entry condition. No significant difference was found for the learning gain on questions testing comprehensive regex skills between the two conditions, but the micro Parsons group had a significantly higher learning gain on multiple choice questions which tested understanding of regex characters.

Many novice programmers struggle to write code from scratch and get frustrated when their code does not work. Parsons problems can reduce the difficulty of a coding problem by providing mixed-up blocks that the learner assembles in the correct order. Parsons problems can also include distractor blocks that are not needed in a correct solution, but which may help students learn to recognize and fix errors. Evidence indicates that students find Parsons problems engaging, easier than writing code from scratch, useful for learning patterns, and typically faster to solve than writing code from scratch with equivalent learning gains. This working group leverages the work of the 2022 ITiCSE working group which published an extensive literature review of Parsons problems and designed and piloted several studies based on the gaps identified by the literature review. The 2023 working group is revising, conducting, and creating new studies. We will analyze the data from these multi-institutional and multi-national studies and publish the results as well as recommendations for future working groups.

Novice programmers struggle with writing code from scratch. One possible way to help them is by using an equivalent Parsons problem on demand, where learners place mixed-up code blocks in the correct order. In a classroom study with 89 undergraduate students, we examined how using a Parsons problem as scaffolding impacts performance and problem-solving efficiency. Results showed that students in the Parsons as Help group achieved significantly higher practice performance and problem-solving efficiency than students who wrote code without help, while achieving the same level of posttest scores. These results improve the understanding of Parsons problems and contribute to the design of future coding practices.

To recognize the current and future trends and challenges in computer science education educational materials for the next decade, the authors of this work provide a conversation to voice the computer science community's experience and expertise on these trends. One of the biggest challenges for introductory computing courses in the next few years will be leveraging the new capabilities of Artificial Intelligent systems such as Open AI CodeX and GPT3 that can generate code from a textual description, explain code, and translate code between programming languages. These tools could drastically change how introductory programming is taught by allowing students to focus more on understanding code, modifying code, and testing code than on writing code. Learning content is increasingly shifting from paper textbooks to online learning systems, which include not just traditional text and figures, but increasingly use interactive items to provide students with better explanations and illustrations, extensive practice, and frequent immediate formative feedback, typically at a lower cognitive load than classical programming assignment. We will discuss challenges and opportunities for interoperability with publishing and learning management platforms. Another example is how guided-based instruments, such as peer team learning, open educational resources, or workbooks, are adaptive and hybrid according to students' needs.

This workshop will introduce computer science teachers (both secondary and post-secondary) to Runestone ebooks and highlight some exciting new features including support for Peer Instruction and new types of Parsons problems. Runestone Interactive (See http://runestone.academy) is an open-source platform for free ebooks. The ebooks support coding in the browser, code visualization/stepping, and a wide variety of practice activities, including multiple-choice, fill-in-the-blank, and mixed-up code (Parsons) problems. The Parsons problems are adaptive, which means that the difficulty of a problem is based on a learner's performance. Over 30 free textbooks are available including ones for CS1, CS2, web development, data science, discrete math, and Advanced Placement (AP) CS (both courses). The presenters have several years of experience creating and using Runestone ebooks. Several studies have demonstrated good usability and positive learning and attitude impacts on students using these ebooks. Participants will gain hands-on experience with the interactive features. Participants will learn how to use the instructor tools which allow them to create a custom course, enroll students in a custom course, create assignments, grade assignments, view student progress, and optionally require use of a spaced practice tool. Participants will also try out the new support for Peer Instruction and new types of Parsons problems. Laptop Required.

Computing culture and its artifacts tend to oust marginalized students. Therefore alongside our push to broaden participation in computing (BPC), we must create new methods of critiquing and changing computing culture and its artifacts. Many marginalized students have experienced what Dr. Ruha Benjamin calls "glitches": a breakdown of computing artifacts regarding the intersection of identities and computing that expose systemic biases. As many egregious "glitches" border on parody or satire, we investigated the potential of satirical programming in BPC programs to critique computing artifacts. We designed and conducted a one-hour session for three high school BPC programs in the American Midwest. Sessions used culturally responsive pedagogy to discuss how programming encodes bias. We taught elements of Python to scaffold the creation of a short Python script. We showed an example of a satirical Python script and encouraged students to satirize a "glitch" they experienced in a computing artifact or concept. Our findings show that many marginalized students were well aware of "glitches" in their own experiences with software. They enjoyed translating the "glitches" into satirical Python scripts that demonstrated their understanding of how systemic biases manifest in software development. In this poster, we share the results of a post-session survey, student examples of satirical scripts, and recommendations for instructors looking to include creative methods of discussing systemic biases in software.

In this paper, we explore using Parsons problems to scaffold novice programmers who are struggling while solving write-code problems. Parsons problems, in which students put mixed-up code blocks in order, can be created quickly and already serve thousands of students while other types of programming support methods are expensive to develop or do not scale. We conducted two studies in which novices were given equivalent Parsons problems as optional scaffolding while solving write-code problems. We investigated when, why, and how students used the Parsons problems as well as their perceptions of the benefits and challenges. A think-aloud observational study with 11 undergraduate students showed that students utilized the Parsons problem before writing a solution to get ideas about where to start; during writing a solution when they were stuck; and after writing a solution to debug errors and look for better strategies. Semi-structured interviews with the same 11 undergraduate students provided evidence that using Parsons problems to scaffold write-code problems helped students to reduce the difficulty, reduce the problem completion time, learn problem-solving strategies, and refine their programming knowledge. However, some students found them less useful if the Parsons solution did not match their approach or if they did not understand the solution. We then conducted a between-subjects classroom study with 81 undergraduate students to investigate the effects on learning. We found that students who received Parsons problems as scaffolding during write-code problems spent significantly less time solving those problems. However, there was no significant learning gain in either condition from pretest to posttest. We also discuss the design implications of our findings.

Programming is a complex task that requires the development of many skills including knowledge of syntax, problem decomposition, algorithm development, and debugging. Code-writing activities are commonly used to help students develop these skills, but the difficulty of writing code from a blank page can overwhelm many novices. Parsons problems offer a simpler alternative to writing code by providing scrambled code blocks that must be placed in the correct order to solve a problem. The extensive literature on Parsons problems documents numerous benefits to using them as both formative and summative assessments. These include more efficient learning, the possibility to dynamically adapt to learner needs, and more reliable grading. Despite these positive findings, further research is needed in order to draw broader inferences. Most work has been conducted at single institutions under unique conditions that are not easily replicated, and some prior studies have been inconclusive or had limitations that affected data validity. To address this, we propose a multi-institutional and multi-national study of the effectiveness of Parsons problems for novice programmers. We will focus on introductory programming courses (CS0/1/2) that use Java, Python, and C/C++ as these are the most common teaching languages. The working group will collaborate to refine the scope, methodology and research questions, and contribute to data collection and analysis.

Adaptive Parsons problems could be used to reduce the difficulty of introductory programming courses and increase the use of active learning in lecture. Parsons problems provide mixed-up code blocks that must be placed in order. In adaptive Parsons problems, if a learner is struggling to solve a problem it can dynamically be made easier. This makes it possible for students to correctly solve a problem in a limited time, even if they are struggling. Previous research on the effectiveness and efficiency of solving Parsons problems for learning has been conducted in controlled conditions or in lab/discussion. We tested the efficiency of solving adaptive Parsons problems versus writing the equivalent code as lecture assignments through three between-subjects experiments. The median time to solve each Parsons problem was less than the median time to write the equivalent code for all but two of the problems, both with complex conditionals. However, that difference was significant for only six of the 10 problems. Our hypothesis for why two problems had a higher median time to solve as a Parsons problem than as a write code problem was that the problem instructions did not match the Parsons problem solution and/or they also had a large number of possible correct solutions. Results from student surveys also provided evidence that most students (78%) find solving adaptive Parsons problems in lecture helpful for their learning, but that some (36.2%) would rather write the code themselves. These findings have implications for how to best use Parsons problems.

Sisters Rise Up is a near-peer mentoring program that attempts to help women of color (Black, Latina, and/or Native American) succeed in their Advanced Placement (AP) Computer Science (CS) courses and on the exams. Women of color are underrepresented in computing careers and have lower pass rates on the AP CS exams than their White and Asian peers. AP Computer Science A (CSA) is equivalent to a college-level first course for computer science majors while AP Computer Science Principles (CSP) is equivalent to an introductory course for non-majors. Secondary students (ages 14-19) take AP courses for college credit and/or placement. Sisters Rise Up offers remote help sessions run by near-peer undergraduate students and uses free and interactive ebooks. Sisters Rise Up originated at the Georgia Institute of Technology in 2014. This paper reports on an effort to scale Sisters Rise Up to six institutions during the 2019-2020 academic year as well as an effort to support women of color through a single institution during 2020-2021. In 2019-2020 there were problems with getting Institutional Review Board (IRB) approval, recruiting, training, and program fidelity. However, at least two of the six institutions had a high pass rate on the AP CS exams in 2019-2020. In 2019-2020 there were 17 women of color enrolled from five states, while in 2020-2021 Sisters Rise Up had 19 women of color enrolled from 12 states. This paper reviews the history, outlines the program details and rationale, presents the outcomes, and outlines future directions.

CSAwesome is a Java AP CSA and CS1 curriculum with 20,000 users on the Runestone ebook platform. The curriculum is online, free and interactive with embedded Java Active Code examples and problems, mixed-up code (Parsons problems), multiple-choice problems, and scaffolded coding challenges. There are many features of the Runestone platform that scaffold and differentiate learning for students. The curriculum is designed to broaden participation in CS and transition students from AP CSP (or CS0) to AP CSA (or CS1) with a variety of techniques such as scaffolded interactivity and creative and collaborative learning. Initial results from the 2020-2021 school year show average gains of 29% on the pre/post test built into the curriculum (n=958, P<.001). Pre and post surveys built into the ebook show slight gains in confidence in Java programming and pursuing further study or a career in computing (P<.001). Female students (22% of those who answered) performed similarly to all students. Students who took AP CSP (39%) prior to AP CSA performed slightly higher in the pre-test but had similar results otherwise. 47% of students rated themselves as beginner programmers and 30% intermediate at the beginning of the course; at the end of the course, 12% rated themselves as beginners and 43% as intermediate programmers.

This workshop will introduce computer science teachers (both secondary and post-secondary) to Runestone ebooks and highlight some exciting new features including support for Peer Instruction, choice questions (choose which questions to answer from several), and exam generation (generate an exam from a list of concepts). Runestone Interactive (See http://runestone.academy) is an open-source platform for free ebooks. The ebooks support coding in the browser, code visualization/stepping, and a wide variety of practice activities, including multiple-choice, fill-in-the-blank, and mixed-up code (Parsons) problems. The Parsons problems are adaptive, which means that the difficulty of a problem is based on a learner's performance. Over 30 free textbooks are available for CS1, CS2, web development, data science, math, and Advanced Placement (AP) CS (both courses). The presenters have several years of experience creating and using Runestone ebooks. Several studies have demonstrated good usability and positive learning and attitude impacts on students using these ebooks. Participants will gain hands-on experience with the interactive features. Participants will learn how to use the instructor tools which allow them to create a custom course, enroll students in a custom course, create assignments, grade assignments, view student progress, and optionally require use of a spaced practice tool. Participants will also try out the new features. Laptop Required.

Ebooks on the Runestone platform contain instructional material (text, images, videos, and a code visualizer/stepper) and a variety of practice problem types (write code problems with unit tests, multiple-choice questions, mixed-up code problems, etc.). User interaction is timestamped and logged. This paper reports on analyses comparing student interaction data to midterm scores for CSAwesome: a College Board endorsed ebook for the Advanced Placement Computer Science A course. We also analyzed mixed-up code (Parsons) problem data in-depth since these are a newer type of practice. Our analysis found that the percent correct on the midterm was most negatively correlated with being in a larger class and most positively correlated with the percent correct on other multiple-choice questions. It was also positively correlated with several other activities including the percent correct on Parsons problems, active code, and the pretest. Interestingly, it was positively correlated with the number of videos viewed, but negatively correlated with the number of videos completed. Next, our analysis of adaptive mixed-up code (Parsons) problems, where the student can ask for help when stuck, found a positive correlation with the number of steps a user completed before asking for help and a negative correlation with the elapsed time before getting help. Looking closely at two Parsons problems, we found that solving each problem more efficiently, i.e., with fewer extra steps, correlated with higher midterm scores. This work could help instructors identify and support struggling students early in a semester and informs the redesign of the instructor dashboard.

Conversational programmers want to learn about code primarily to communicate with technical co-workers, not to develop software. However, existing instructional materials don’t meet the needs of conversational programmers because they prioritize syntax and semantics over concepts and applications. This mismatch results in feelings of failure and low self-efficacy. To motivate conversational programmers, we propose purpose-first programming, a new approach that focuses on learning a handful of domain-specific code patterns and assembling them to create authentic and useful programs. We report on the development of a purpose-first programming prototype that teaches five patterns in the domain of web scraping. We show that learning with purpose-first programming is motivating for conversational programmers because it engenders a feeling of success and aligns with these learners’ goals. Purpose-first programming learning enabled novice conversational programmers to complete scaffolded code writing, debugging, and explaining activities after only 30 minutes of instruction.

Novice programmers need differentiated assessments (such as adaptive Parsons problems) to maximize their ability to learn how to program. Parsons problems require learners to place mixed-up code blocks in the correct order to solve a problem. We conducted a within-subjects experiment to compare the efficiency and cognitive load of solving adaptive Parsons problems versus writing the equivalent (isomorphic) code. Undergraduates were usually more significantly efficient at solving a Parsons problem than writing the equivalent code, but not when the solution to the Parsons problem was unusual. This has implications for problem creators. This paper also reports on the mean cognitive load ratings of the two problem types and the relationship between efficiency and cognitive load ratings. Lastly, it reports on think-aloud observations of 11 students solving both adaptive Parsons problems and write-code problems and the results from an end-of-course student survey.

Runestone Interactive is an open-source ebook platform designed to create and publish free interactive computer science textbooks. (See http://runestoneinteractive.org). Runestone ebooks support programming within the browser, code visualization/stepping, and a variety of practice activities, including multiple choice, fill-in-the-blank, clickable-code, and mixed-up code (Parsons problems). Over 20 free textbooks have been created for CS1, CS2, web development, data science, and AP CS (both courses). The presenters have several years of experience creating and using Runestone ebooks. Several studies have demonstrated good usability and positive learning and attitude impacts on students using these ebooks. Over 25,000 people a day use Runestone ebooks. This workshop will introduce computer science teachers (both secondary and post-secondary) to Runestone ebooks. The hands-on session will start by leading participants through use of the ebooks as if they were students. Participants will be introduced to the interactive features of the ebooks. Participants will next create a custom course from an existing ebook and will use the instructor's dashboard to review student activity, modify the course, grade students, and set up a spaced practice tool. Participants will create their own assignments using existing problems. Participants will also be shown how to create new material for assignments, such as multiple-choice questions for a quiz. Laptop Required.

This hands-on online workshop will introduce high school and college instructors to CSAwesome, a free Java curriculum and ebook at course.csawesome.org for the Advanced Placement (AP) Computer Science (CS) A course. This course is equivalent to a college-level CS1 course in Java. CSAwesome is an official College Board approved curriculum and professional development provider and has been widely adopted by AP high school teachers.

The Runestone platform is open-source, extensible, and serves free ebooks to over 25,000 learners a day from around the world. The site hosts 18 ebooks for computing courses. Some of these ebook have been translated into several languages. There are ebooks for secondary computer science (AP CSP and AP CSA), CS1, CS2, data science, and web programming courses. The platform currently supports executable and editable examples in Python, Java, C, C++, HTML, JavaScript, Processing, and SQL. Runestone provides features for instructors, learners, authors, and researchers. Instructors can create a custom course from any of the existing ebooks and their students can register for that course. Instructors can create assignments from the existing material or author new problems, grade assignments, and visualize student progress. Learners can execute and modify examples and answer practice questions with immediate feedback. Runestone includes common practice types, such as multiple-choice questions, as well as some unique types, such as adaptive Parsons problems. Authors can modify the existing ebooks or write new ebooks using restructuredText: a markup language. Researchers can create and test new interactive features, run experiments, and analyze log file data. This paper describes the architecture of the platform, highlights some of the unique features, provides an overview of how instructors use the platform, summarizes the research studies conducted on the platform, and describes plans for future development.

Runestone Interactive is an open-source ebook platform (http://runestoneinteractive.org). Runestone supports executable and editable examples in Python, Java, C, C++, JavaScript, HTML, Processing, and SQL. It also includes code visualization/stepping and a wide variety of practice activities, including multiple-choice, fill-in-the-blank, clickable-code, and mixed-up code (Parsons) problems. A new practice tool reuses ebook content to provide spaced, retrieval-based, and interleaved practice. Free textbooks have been created for CS1, CS2, web development, data science, and Advanced Placement (AP) CS (both CSP and CSA). Over 25,000 people a day use Runestone ebooks. Several studies have demonstrated positive learning and attitude outcomes from the use of interactive ebooks versus static ebooks.

Retrieval practice, spacing, and interleaving are known to enhance long-term learning and transfer, but reduce short-term performance. It can be difficult to get both students and instructors to use these techniques since they perceive them as impeding initial student learning. We leveraged user experience design and research techniques, including survey and participant observation, to improve the design of a practice tool during a semester of use in a large introductory Python programming course. In this paper, we describe the design features that made the tool effective for learning as well as motivating. These include requiring spacing by giving credit for each day that a student answered a minimum number of questions, adapting a spaced repetition algorithm to schedule topics rather than specific questions, providing a visual representation of the evolving schedule in order to support meta-cognition, and providing several gameful design elements. To assess effectiveness, we estimated a regression model: each hour spent using the practice tool over the course of a semester was associated with an increase in final exam grades of 1.04%, even after controlling for many potential confounds. To assess motivation, we report on the amount of practice tool use: 62 of the 193 students (32%) voluntarily used the tool more than the required 45 days. This provides evidence that the design of the tool successfully overcame the typically negative perceptions of retrieval practice, spacing, and interleaving.

Prior research has shown that sketching out a code trace on paper is correlated with higher scores on code reading problems. Why do students sometimes choose not to draw out a code trace, or if they do, choose a different sketching technique than their instructor has demonstrated? In this study, we interviewed 13 CS1 students retrospectively about their decisions to sketch and draw on a recent programming exam. When students do sketch, we find that their sketching choices do not always align with a strict execution of the notional machine. Sketching choices are driven by a search for a program's patterns, an attempt to create organizational structure among intermediate values, and the tracking of prior steps and results. When novices don't sketch, they often report that they've identified the goal that the code achieves. In either case, novices are searching for the functionality of code, rather than merely tracing its behavior.

Many researchers [1] recognize that great Computer Science (CS) teaching demands great pedagogy, technology, and subject knowledge. It is believed that without a focus on teachers, which are the connection between schools, universities, and work, and their preparation the gap in CS knowledge will be difficult to close [2] and the support to curricular indications, as the national curriculum for computing in England, Computer Science Principles, CS 10K, CS for All, CSTA, Scientix, will not sustainable. World-spread projects as Computing At School, Mobile Computer Science Principles or Initial Teacher Education Lab emphasize the role of teachers in reaching students, sustaining their daily practice. International initiatives as code.org, CoderDojo, Code Club are great examples of well propagated curricular, pedagogical and technological innovations in education. Beside this the research effort has produced many block-based programming languages, tools, and technologies "Low floor and high ceiling". With this deluge of content, pedagogical and technological innovations there is the necessity to involve and sustain teachers in getting confidence with CS. Ways to accomplish this will be discussed.

Runestone Interactive is an open-source ebook platform designed to create and publish interactive computer science textbooks. (See http://runestoneinteractive.org). Runestone ebooks support programming in the browser, code visualizations, and a wide variety of practice activities, from multiple choice and fill-in-the-blank questions to Parsons Problems (drag-and-drop mixed-up code). Free textbooks have been created for CS1, CS2, and Advanced Placement (AP) CS. The presenters have several years of experience creating and using Runestone ebooks. Several studies have demonstrated good usability and positive learning and attitude impacts on students using these ebooks. Runestone ebooks can be customizable to meet the needs of individual courses and teachers. Over 20,000 people a day use Runestone ebooks. The goal of this workshop is to introduce computer science teachers (both secondary and post-secondary) to Runestone ebooks. This hands-on session will start by leading participants through use of the ebooks as if they were students. Participants will try each of the interactive features. They will also create a custom course from an existing ebook, use the instructor dashboard to review student activity, modify the course, and grade students. Participants will create their own assignments using existing active learning components. They will also be shown how to create new material for assignments, such as multiple-choice questions for a quiz. Laptop Required.

Efforts to improve and promote CS education characterized by greater participation of underrepresented groups have taken off sharply in the six years since the Expanding Computing Education Pathways (ECEP) NSF alliance was first funded. Though many of these initiatives are occurring at the student and teacher level, ECEP and other national-level organizations have demonstrated the importance of focusing on change at the state level, as this can have important, long-term implications on policy and infrastructure. This poster describes ECEP's emergent design as a backbone organization helping state-level teams achieve educational change around CS. It also highlights key lessons learned through the alliance about how to foster CS education change at the state level. Four key lessons stand out. 1) The importance of understanding that engaging states requires appreciating the commonality of a framework around key areas to change (such as state standards, teacher certification and graduation requirements) but also respecting the uniqueness of state circumstances in these change areas. 2) The shared value created through a collective impact model to build a community that can support rich discussion and sharing of ideas. 3) The utility of an empirically derived a 4-stage model of state change used by ECEP that emphasized defining leaders, understanding the CS education landscape and finding allies to support efforts. and 4) The role of state team structures that can facilitate change when they are guided by identifying and engaging key stakeholders within a state and giving them a voice in driving the strategies and plans.

Practice is essential for learning. There is evidence that solving Parsons problems (putting mixed up code blocks in order) is a more efficient, but just as effective, form of practice than writing code from scratch. However, not all students successfully solve every Parsons problem. Making the problems adaptive, so that the difficulty changes based on the learner's performance, should keep the learner in Vygotsky's zone of proximal development and maximize learning gains. This paper reports on a study comparing the efficiency and effectiveness of learning from solving adaptive Parsons problems vs non-adaptive Parsons problem vs writing the equivalent code. The adaptive Parsons problems used both intra-problem and inter-problem adaptation. Intra-problem adaptation means that if the learner is struggling to solve the current problem, the problem can dynamically be made easier. Inter-problem adaptation means that the difficulty of the next problem is modified based on the learner's performance on the previous problem. This study provides evidence that solving intra-problem and inter-problem adaptive Parsons problems is a more efficient, but just as effective, form of practice as writing the equivalent code.

Rise Up 4 CS was created at Georgia Tech in the spring of 2013 to help African American students succeed in their Advanced Placement (AP) Computer Science A (CSA) high school course and on the exam. The AP CSA course is intended to be equivalent to a college level CS1 course. In the fall of 2014, Sisters Rise Up 4 CS was also created to help female students succeed. Rise Up (RU) and Sisters Rise Up (SRU) both offer remote and in-person help sessions led by undergraduate students, who serve as near-peer role models. The long-term goal of these projects is to attract more underrepresented students to computing careers by increasing their self-efficacy. Self-efficacy is the belief that one can succeed in a particular task or field. Thanks to generous support from Google, RU and SRU have been offered at several other colleges and universities as well. This paper summarizes the results from spring 2013 to summer 2016, reports on an alumni survey answered by 68 (32%) of the 211 alumni, and includes excerpts from five semi-structured interviews. The majority (63%) of the alumni survey respondents, who are in college, are majoring in computing. A similar percentage of the alumni who are still in high school intend to major in computing (62%). When asked about the programs' impact on their career choices and interest in computing, 61% of respondents indicated that the program increased their interest in computer science and 24% indicated that it changed their career plans to computing.

Sketching out a code trace is a cognitive assistance for programmers, student and professional. Previous research (Lister et al. 2004) showed that students who sketch a trace on paper had greater success on code 'reading' problems involving loops, arrays, and conditionals. We replicated this finding, and developed further categories of student sketching strategies. Our results support previous findings that students who don't sketch on code reading problems have a lower success rate than students who do sketch. We found that students who sketch incomplete traces also have a low success rate, similar to students who don't sketch at all. We categorized sketching strategies on new problem types (code writing, code ordering, and code fixing) and find that different types of sketching are used on these problems, not always with increased success. We ground our results in a theory of sketching as a method for distributing cognition and as a demonstration of the process of the notional machine.

Online education is an important tool for supporting the growing number of teachers and students in computer science. We created two eBooks containing interactive content for Advanced Placement Computer Science Principles, one targeted at teachers and one at students. By comparing the eBook usage patterns of these populations, including activity usage counts, transitions between activities, and pathways through the eBook, we develop a characterization of how student use of the eBook differs from teacher use. We offer design recommendations for how eBooks might be developed to target each of our populations. We ground our recommendations in a theory of teachers as expert learners who possess a greater ability to regulate their own learning process.

To reach President Obama's vision of computer science for all, we will need to prepare many more secondary teachers to teach computer science. In 2015, less than 3,000 schools passed the College Board's audit to offer the Advanced Placement (AP) Computer Science (CS) A course, while close to 12,000 schools passed the audit to offer Calculus AB. The presenters have led teacher professional development workshops for many years and will share their knowledge and materials to help others offer effective teacher professional development. In particular they will cover how to plan a workshop, how to find funding, how to prepare materials for secondary teachers, what materials are available for teacher professional development, how to teach pedagogical content knowledge (how to teach computer science), and how to increase diversity in computer science classrooms. The presenters have free materials for the new Advanced Placement Computer Science Principles (CSP) course as well as the Advanced Placement Computer Science A course. These materials include free interactive ebooks for both CSP teachers and students to help them learn programming and a free interactive ebook for the AP CS A course. Participants will be encouraged to share their materials and strategies as well. Laptop Required.

In January 2016, US President Barack Obama started an initiative to provide CS for All -- with the goal that all school students should have access to computing education. Computing departments in higher education have a particularly important role to play in this initiative. It's in our best interest to get involved, since the effort can potentially improve the quality of our incoming students. CS Departments have unique insights as subject-matter experts to inform the development of standards. We can provide leadership to inform and influence education policy. In this session, we will present a variety of ways in which departments and faculty can support CS for All and will answer audience questions about the initiative. Our goal is to provide concrete positive actions for faculty.

Runestone is an open-source ebook platform designed to create and publish interactive computer science textbooks (See http://runestoneinteractive.org/). Runestone textbooks support programming within the browser, code visualizations, and a wide variety of practice activities, from multiple choice and fill-in-the-blank questions to Parsons Problems (drag-and-drop mixed-up code). The presenters have several years of experience developing and using ebooks for CS1, AP CS A, AP CSP, data structures, and web programming. Several studies have demonstrated good usability and positive learning and attitude impacts on students using these ebooks. Runestone ebooks can be customized to meet the needs of individual courses and teachers. The goal of this workshop is to help computer science teachers use and modify Runestone ebooks. The hands-on session will start by leading participants through use of the ebooks as if they were students. Participants will next create their own custom course of any existing ebook in the library and will use the instructor's dashboard to review student activity, modify the course, and grade students. Finally, participants will create their own assignments using Runestone's active learning components, which serves as a starting point for authoring their own content in Runestone. Laptop Required.

Parsons problems are code segments that must be placed in the correct order with the correct indention. Research on Parsons problems suggests that they might be a more effective and efficient learning approach than writing equivalent code, especially for time-strapped secondary teachers. I am exploring this hypothesis with empirical experiments, observations, and log file analyses. Our research team also plans to modify the open-source js-parsons software to allow Parsons problems to be dynamically adaptive, which means that the difficulty of the problem will change based on the user's past performance. I plan to compare dynamically adaptive Parsons problems to the current non-adaptive Parsons problems to determine which learners prefer and to see if solving dynamically adaptive Parsons problems leads to more efficient and effective learning than solving non-adaptive Parsons problems.

Several countries are trying to provide access to computing education for all secondary students. However, there are not enough teachers who are prepared to teach computer science. Interactive electronic books (ebooks) are a promising approach for providing low-cost professional development in computer science. Over the last four years, our research group has been conducting design-based research by iteratively developing and testing versions of a teacher ebook to help secondary teachers with no programming experience learn to teach an introductory programming course. The interactive elements in the ebook were designed based on research results from educational psychology and are intended to make learning more efficient and effective. Our goals for this effort are to increase teachers' knowledge of computer science concepts and to improve teachers' confidence in their ability to teach computer science. In this paper we summarize our previous work and report on a large-scale study of version two of the teacher ebook. We also recommend several design principles for interactive ebooks for computing teachers based on feedback from teachers, log file analyses, and randomized controlled studies.

Female students are underrepresented in computing careers and in computing majors at the college or university level. This underrepresentation starts in high school. Only 20% of Advanced Placement (AP) Computer Science (CS) A exam takers in 2014 nationally were female. In addition to being underrepresented, female students tend to have a lower pass rate on the exam than male students. Failing the exam could reduce interest in computing as a career. Sisters Rise Up 4 CS was created at the Georgia Institute of Technology (Georgia Tech) to help more female high school students pass the AP CS A exam. It offers help sessions, near-peer role models, exposure to a college campus, and a community of learners. Sisters Rise Up 4 CS is based on Project Rise Up 4 CS, which was also created at Georgia Tech to help increase the number of African-American students that pass the AP CS A exam. However, Sisters Rise Up 4 CS does not include the financial incentives offered by Project Rise Up 4 CS. Project Rise Up 4 CS has resulted in statistically significant positive changes in attitudes for participants and a large increase in the participants' perception of their ability to pass the exam. Even students who did not pass the exam have reported positive benefits from being in Project Rise Up 4 CS. This paper summarizes the findings from Project Rise Up 4 CS from 2013-2015 and presents the findings from the pilot of Sisters Rise Up 4 CS from 2014-2015. We used a mixed-methods approach to evaluate the project and to judge the effectiveness of the project elements.

We report on a study that used subgoal labels to teach students how to write while loops with a Parsons problem learning assessment. Subgoal labels were used to aid learning of programming while not overloading students' cognitive abilities. We wanted to compare giving learners subgoal labels versus asking learners to generate subgoal labels. As an assessment for learning we asked students to solve a Parsons problem -- to place code segments in the correct order. We found that students who were given subgoal labels performed statistically better than the groups that did not receive subgoal labels or were asked to generate subgoal labels. We conclude that a low cognitive load assessment, Parsons problems, can be more sensitive to student learning gains than traditional code generation problems.

Parsons problems are code segments that must be placed in the correct order with the correct indention. Research on Parsons problems suggests that they might be a more effective and efficient learning approach than writing equivalent code, especially for time-strapped secondary teachers. I intend to explore this hypothesis with empirical experiments, observations, and log file analysis.

Educational psychology findings indicate that active processing (such as self-testing) is more effective for learning than passive reading or even rereading. Electronic books (ebooks) can include much more than static pictures and text. Ebooks can promote better learning by increasing the reader's interaction with the material through multi-modal learning supports, worked examples, and low cognitive load practice activities. For example, multiple choice questions with immediate feedback can help identify misconceptions and gaps in knowledge. Parsons problems, which are mixed up code segments that have to be put in the correct order, require learners to think about the order of the statements in a solution without having to worry about syntax errors. Our research group has been applying concepts from educational psychology to make learning from ebooks more effective and efficient. This paper reports on an observational study and log file analysis on the use of an ebook that incorporates interactive activities. We provide evidence that learners engaged in the interactive activities, but used some types of activities more than others. We also found evidence that learners encountered some "desirable difficulties" which can improve learning. This descriptive study informs a research agenda to improve the quality of instruction in computing education.

We are all excited to see an increasing number of national efforts to reform computing education. Persistent and sustainable change in a state, its higher education institutions, and its K12 districts requires the active participation of individuals and local organizations to engage policy makers, higher education, school districts and communities. U.S. education is highly distributed, with critical decisions pushed more to the community level and less at the national (or even state) level. The system is organized along K20 pathways and a reform process to improve these pathways will take place at multiple levels and state by state.

This workshop will provide details on how to plan and run non-residential computing summer camps for 4th -- 12th grade students. Georgia Tech has been offering computing summer camps since 2004. These camps are financially self-sustaining and effective. Items used in past camps include: CS Unplugged, LightBot, Scratch, Alice, LEGO robots (WeDo, NXT, EV3, and Tetrix), EarSketch, and App Inventor. Georgia Tech helped start or expand other computing camps at eleven other colleges and universities in Georgia from 2007 to 2010 as part of Georgia Computes!. This last year as part of the Expanding Computing Education Pathways (ECEP) NSF grant we have also helped institutions start or expand summer computing camps in South Carolina, Massachusetts, and California.

The recent successes of Computer Science Education Week and code.org's Hour of Code have meant that more K-12 students than ever are being given an authentic, engaging and eye-opening exposure to the wonders of computer science. There are resources aplenty to help high school and college faculty with outreach. These range from easy-to-learn, open-ended programming environments (Scratch, Alice, Snap!), to online coding challenges (code.org, Lite-bot), to non-computer activities with live performances (CS Unplugged, cs4fn), to having the entire outreach experience delivered "in a box", thanks to NCWIT.

Parsons problems are code segments that the user must put in the correct order with the correct indention. Research on Parsons problems suggests that they might be a more effective and efficient learning approach than writing equivalent code, especially for time-strapped secondary teachers. I intend to explore this hypothesis with empirical experiments, observations, and log file analysis.

This paper describes Project Rise Up 4 CS, an attempt to increase the number of Black students in Georgia that pass the Advanced Placement (AP) Computer Science (CS) A exam. In 2012 Black students had the lowest pass rates on the AP CS A exam both in Georgia and nationally. Project Rise Up 4 CS provided Black students with role models, hands-on learning, competitions, a financial incentive, and webinars on AP CS A content. The first cohort started in January of 2013 and finished in May 2013. Of the 27 students who enrolled in the first cohort, 14 met all of the completion requirements, and 9 (69%) of the 13 who took the exam passed. For comparison, in 2012 only 22 (16%) of 137 Black students passed the exam in Georgia. In 2013, 28 (22%) of 129 Black students passed the exam in Georgia. This was the highest number of Black students to pass the AP CS A exam ever in Georgia and a 27% increase from 2012. In addition, students who met the completion requirements for Project Rise Up 4 CS exhibited statistically significant changes in attitudes towards computing and also demonstrated significant learning gains. This paper discusses the motivation for the project, provides project details, presents the evaluation results, and future plans.

We examine the current state of computing education in the United States, in order to be able to identify problems in diversity and performance. Data on computing education are difficult to come by, since computer science courses are not tracked in US public education systems. By using a large and nationwide quantitative data source, we can gain new insights into who is participating in computing education, where the greatest need is, and what factors explain the variance between states. We used data from the Advanced Placement Computer Science A (AP CS A) exam to get a detailed view of the demographics of who is taking the exam across the United States and in each state, and how they are performing on the exam. We use economic and census data to provide explanations for some of the AP CS data. We find that minority group involvement is low in AP CS A, but the variance between states in terms of exam-takers is driven by minority group involvement. We find that wealth in a state influences the number of students taking the AP CS A exam, but indirectly.

This workshop will provide details on the projects that we recommend for non-residential computing summer camps for 4th -- 12th grade students. Georgia Tech has been offering computing summer camps since 2004. These camps are financially self-sustaining and effective at improving attitudes towards computing, especially for females and African-American students. Items used in the camps include: CS Unplugged, LightBot, Scratch, Alice, LEGO robots (WeDo, NXT, and Tetrix), EarSketch, and App Inventor. Georgia Tech has also helped eleven other colleges and universities in Georgia start or expand computing summer camps. Some of these other institutions also offer camps on GameSalad, GameMaker, and web development. These other camps have also improved attitudes towards computing. The workshop will include an overview of the projects that we have found to be successful and include hands-on work with App Inventor, GameSalad, LEGO NXT robots, and LEGO EV3 robots. laptop optional

Georgia Computes! ( GaComputes ) was a six-year (2006--2012) project to improve computing education across the state of Georgia in the United States, funded by the National Science Foundation. The goal of GaComputes was to broaden participation in computing and especially to engage more members of underrepresented groups which includes women, African Americans, and Hispanics. GaComputes’ interventions were multi-faceted and broad: summer camps and after-school/weekend programs for 4th--12th grade students, professional development for secondary teachers, and professional development for post-secondary instructors faculty. All of the efforts were carefully evaluated by an external team (led by the third and fourth authors), which provides us with an unusually detailed view into a computing education intervention across a region (about 59K square miles, about 9.9 million residents). Our dataset includes evaluations from over 2,000 students who attended after-school or weekend workshops, over 500 secondary school teachers who attended professional development, 120 post-secondary teachers who attended professional development, and over 2,000 students who attended a summer day (non-residential) camp. GaComputes evaluations provide insight into details of interventions and into influences on student motivation and learning. In this article, we describe the results of these evaluations and describe how GaComputes broadened participation in computing in Georgia through both direct interventions and indirect support of other projects.

This workshop will provide details on how to plan and run non-residential computing summer camps for 4th -- 12th grade students. Georgia Tech has been offering computing summer camps since 2004. These camps are financially self-sustaining and effective. Items used in the camps include: CS Unplugged, LightBot, Scratch, Alice, LEGO robots (WeDo, NXT, and Tetrix), EarSketch, and App Inventor. Georgia Tech has also helped start other computing camps at eleven other colleges and universities in Georgia.

In computing education, we have only just started developing methods for accurately measuring a student's understanding of introductory computing, let alone characterizing a whole classroom, school, or university system. As part of evaluating the impact of "Georgia Computes!" we sought an understanding the factors influencing undergraduate enrollment in introductory computing for an entire state in the United States of America. We gathered surveys from over 1400 undergraduate students in introductory computing classes from 19 higher-education institutions in a single state. The analysis provided insight into the impact of "Georgia Computes!", into the connections between stages of the computing education pathways, and how factors that influence students' pursuit of computing differ between genders and majority/minority group students.

We know that students bring diverse experiences and an assortment of learning styles into our classrooms. We greet them and hand out a syllabus listing the required textbook(s). One size does not fit all when it comes to textbooks. In this session, participants will discover teaching activities that can be used to supplement any computer science textbook. These activities include human modeling and role playing, interactive whiteboard activities, case studies, class projects, games, and more. By exploring "extra" activities not found in most textbooks, participants will learn how these hands-on, interesting and fun activities can help students understand the topics listed in the AP Computer Science A Course Description: Object-oriented Program Design, Program Implementation, Program Analysis, Standard Data Structures, Standard Algorithms, and Computing in Context. Student attention spans seem to be getting shorter. The panelists found themselves spending more and more time searching for and/or developing supplemental resources in efforts to reach and retain students. Members of the AP Computer Science-A Development Committee will share these resources and lead a discussion of proven strategies and lesson ideas for teaching outside the textbook.

universities. But, it is not enough to offer computing summer camps and hope that students like them. The camps should be effective by some measure, such as broadening participation by underrepresented groups and/or increasing learning. Summer camps should also be financially sustainable, so that institutions can continue to offer them regularly. The summer camps at Georgia Tech have become effective and financially sustainable. This paper presents the rationale for our camps, the evaluation results that demonstrate positive attitude changes and increases in learning, and the business model that makes them financially sustainable. It also reports on the evaluation results from seven other colleges and universities in Georgia that offered computing summer camps during the summer of 2011 with assistance from Georgia Tech.

In 2006, computing education was suffering from a crisis - enrollments were dropping sharply at universities and colleges across the United States, and interest in computing from high school and middle school students was waning significantly. At the 2007 SIGCSE Symposium, the ACM Education Board organized a special session to explore the underlying causes. In his keynote at the same conference, Grady Booch exhorted us to share the "passion, beauty, joy and awe" (PBJA) of computing. This led to a series of room-packed sessions at the following four SIGCSE symposia to explore that idea from different angles. They have provided a forum for sharing: "What we've done: Highlighting successful PBJA initiatives the presenters have undertaken or seen and wish to celebrate. "What we should do (curriculum): Pointing out where our curriculum is lacking in PBJA, and how to fix it. "How we should do it (pedagogy): Sharing how a change in attitude / focus / etc. can make strides to improving PBJA.

Everybody teaches linked lists, with homework like implementing duplicate, weave, and reverse. When those nodes contain strings or numbers, these are pretty boring assignments. When these nodes contain music (MIDI), these operations are composing music, which can then be played. This workshop shows how to use music, images, and sounds to teach the basic data structures, including linked lists, circular linked lists, stacks, queues, and trees. These pieces can then be tied together through the use of simulations to generate animated movies. We will be using Java, though many of the methods can also be used in Python.

Much has been written about the decrease in the number of students pursuing computing in colleges. Unfortunately, most students form their (usually negative) opinions of computing long before they reach college [1], so it is beneficial for university faculty who want to increase college enrollment to do outreach aimed at a younger audience. Activities to promote computing among K-12 students can proceed at several levels, occur in various locations, and involve both students and faculty.College and high school faculty can collaborate in many ways. As teachers of introductory courses, both groups want to generate interest, cover important material, and provide a strong foundation for later work. For example, the AP CS course description includes material that fits well with both high school and introductory college courses. Also, today in both environments, there is much discussion regarding the best approaches for creating excitement and introducing students to computing. Although both faculties need to address common content and utilize active pedagogies, it also is important to acknowledge that college and high school environments are different, and both levels have special opportunities and constraints.Many college faculty are interested in doing K-12 outreach but often do not know where to start or do not know any successful strategies. The purpose of this panel is to present K-12 outreach strategies that have worked and then to provide the audience with the opportunity to brainstorm with the panelists to explore alternate ideas for outreach and to identify new approaches. Some ideas and strategies come from experiences of high school and college teachers working together as part of the AP CS program, but other approaches extend beyond AP CS.

The SIGCSE community has been exploring the role of multimedia to enhance computing education since the earliest algorithm visualization systems and studies [1]. Media Computation is a shift in focus [2]. Where algorithm visualization presents information to the student to facilitate their understanding, media computation is about having students manipulate media as the data for their programming, i.e., as the focus of the course activities. Students in media computation produce new images, sounds, and video. We aim to show that computer science is about more than numbers and strings. Computer science is also about creative expression. The original media computation work focused on using media to motivate non-computing majors [2]. The role of media in motivating student learning for computing education has broadened. Inventive teachers are using media computation for lots of different kinds of students, at different kinds of institutions, with a range of languages and toolkits. This special session is a mixture of "Five Minute Madness," science fair, and art gallery. Each participant will present how he or she is using media to motivate student learning, and some student work will be available for audience inspection

Computing education suffers from low enrollment and a lack of diversity. Both of these problems require changes across the entire computing education pipeline. The "Georgia Computes!" alliance, funded by the National Science Foundation's Broadening Participation in Computing program, seeks to improve the computing education pipeline in Georgia. "Georgia Computes!" is having a measurable effect at each stage of the pipeline, but has not yet shown an impact across the whole pipeline.

The Institute for Computing Education (ICE) was created in the spring of 2004. ICE is a partnership between the Georgia Department of Education and the College of Computing at Georgia Tech. The goals for this partnership are to increase the number and quality of computer science teachers and increase the number, quality, and diversity of computer science students. One specific goal is to increase the number of students taking the CS-Advanced Placement (AP) course. In this paper we report on both the progress we have made towards these goals and the problems we have encountered. We hope that other states will create similar partnerships and leverage our experience.

This session will continue efforts by the ACM and Computer Science Teachers Association (CSTA) to create a comprehensive body of resources to support the implementation of a national K-12 computer science curriculum. The ACM K-12 Computer Science Curriculum Committee, with input from the Computer Science educational community, published a Model Curriculum [1] in 2003. The Model suggests a four-level curriculum. Level-I topics, primarily for the K-8 years, closely follow the well-defined ISTE Standards [2]. Level-II topics are suggested for all students in grades 9 or 10 and are comprised of introductory computer science content. Level-III topics focus on analysis and design with emphasis on the scientific and engineering aspects of computer science. The Level III course is intended for students who may be interested in continuing their education in computer science at the college level. AP Computer Science and special topics courses comprise Level-4. This special session will focus on the learning objectives and activities for Level III of the K-12 Computer Science Model curriculum. Attendees will be invited to provide critical feedback that will help move the curriculum project forward.

This paper describes how the Institute for Computing Education (ICE) at Georgia Tech is trying to improve the state of computer science education in secondary schools in Georgia. ICE is a partnership between the Georgia Department of Education and the College of Computing at Georgia Tech. The goals for this partnership are to increase the number and quality of computer science teachers and increase the number, quality, and diversity of computer science students. One specific goal is to increase the number of students taking the CS-AP course. We believe that this partnership can serve as a model for other states.

The development of an intelligent assistance agent for diagnostic radiology faces a number of challenges: e.g., how to gain access to the knowledge used in such visual problem solving tasks, how to operationalize it for further processing, and how to present it in the most useful manner for diagnosis. Our previous work in this domain placed an emphasis on the first two issues, and has shown that both perception and problem solving need to be supported in these types of tasks. However, the interface to the underlying intelligence in our first prototypes did not reflect how medical practitioners physically interact with images in the context of their work. The paper describes current work in progress on an image-centered approach to the agent's interface design. This approach will allow users to interact dynamically with the image, and to store these interactions for future retrieval.