Abstract Large language models (LLMs) are rapidly reshaping software development, but their impact across the full software development lifecycle is underexplored. Existing work tends to focus on isolated activities such as code generation or testing, leaving open questions about how LLMs affect developers, processes, products, and the broader software ecosystem. We address this gap through semi-structured interviews with sixteen early-adopter software professionals who integrated LLM-based tools into their day-to-day work in early to mid-2023. We treat these interviews as early empirical evidence and compare participants’ accounts with recent work on LLMs in software engineering, noting which early patterns persist or shift. Using thematic analysis, we organize our findings around four dimensions: people, process, product, and society. Developers reported substantial productivity gains from reducing mundane tasks, streamlining search, and accelerating debugging, but also described a productivity-quality paradox: they frequently discarded generated code and shifted effort from writing code to critically evaluating and integrating it. LLM use was highly phase-dependent, with strong uptake in implementation and debugging but limited influence on requirements gathering and collaborative work. Participants developed new competencies to use LLMs effectively, including prompt engineering strategies, multi-layered verification, and security-conscious integration to protect proprietary data. They also anticipated changes in hiring expectations, team practices, and computing education, while emphasizing that human judgment and foundational software engineering skills remain essential. Our findings, consistent with evidence from large-scale studies, offer actionable implications for developers, organizations, educators, and tool designers seeking to integrate LLMs responsibly into professional software practice.

As artificial intelligence (AI) becomes increasingly integrated into daily life, there is a critical need for developing AI literacy across all educational levels. However, current AI education remains largely confined to college-level computer science classrooms with limited access for K-12 learners. We present the AI Scholars Program, a novel approach that addresses the AI education gap by preparing college computing students to serve as AI education ambassadors in their communities and empowering K-12 teachers to adopt AI education practices in their classrooms. This experience report presents the curriculum and its outcomes after one round of refinement. The program offers structured AI learning through bi-weekly webinars, resources, and collaborative opportunities to form teams and conduct community outreach projects. Our program invited 63 scholars from 30 institutions across the U.S., including 51 college students and 12 K-12 teachers. Their outreach impacted over 230 K-12 learners. We examine program outcomes for participants and projects through pre/post surveys measuring computing attitudes and self-efficacy for teaching AI, scholar interviews, and outreach project reports. We share lessons learned and challenges for designing similar programs, highlighting the importance of involving educators for effective community-engaged AI education. The program creates a sustainable pipeline for college students to develop technical skills and leadership while addressing K-12 AI education shortages. We contribute insights for scaling AI literacy and broadening participation in computing.

Block-based programming puzzle games are popular as engaging, visual tools for introducing novice learners to programming. Many incorporate 3D environments, where players navigate and solve spatial challenges. However, little research has examined how spatial reasoning skills interact with game features to shape player experience. This study investigates the interplay between game features, spatial ability, in-game performance, and perceived difficulty within BOTs: 3D programming game where players plan spatial movements to solve puzzles. In an online study with 60 players, we examined how feature changes affected performance and perceived difficulty. Spatial skills strongly predicted performance but did not predict perceived difficulty. Larger or more complex layouts increased performance costs, with backward-facing player characters producing the largest spike in performance demands. Loops reliably increased perceived difficulty. Our findings highlight concrete needs for early spatial scaffolds, clearer support for mental-model shifts, and better cues for recognizing repetition and abstraction.

AI-based chatbots have the potential to accelerate learning and teaching, but may also have counterproductive consequences without thoughtful design and scaffolding. To better understand teachers’ perspectives on large language model (LLM) based chatbots, we conducted a study with 11 teams of middle-school teachers using chatbots for a science and computational thinking activity within a block-based programming environment. Based on a qualitative analysis of audio transcripts and chatbot interactions, we propose three profiles: explorer, frustrated, and mixed that reflect diverse scaffolding needs. In their discussions, we found that teachers perceived chatbot benefits such as building prompting skills and self confidence alongside risks including potential declines in learning and critical thinking. Key design recommendations include scaffolding the introduction to chatbots, facilitating teacher control of chatbot features, and suggesting when and how chatbots should be used. Our contribution informs the design of chatbots to support teachers and learners in middle school coding activities.

Children increasingly interact with generative AI systems that can produce hallucinated content, potentially reinforcing misconceptions and undermining critical thinking skills. We investigate how children detect and respond to hallucinations while building and testing LLM-powered chatbots in a development environment. We integrated hallucination-awareness scaffolds such as confidence indicators, fact-checking, repeated questioning, and model comparison. Through a study with 48 middle school learners aged 10-14, participants showed significant pre-to-post gains in AI knowledge, hallucination awareness, and confidence in building trustworthy chatbots. They developed multi-layered strategies, including probing inconsistencies and cross-checking with external sources. Key challenges included over-reliance on visible cues, fragmented use of scaffolds, and a tension between creativity and reliability. These findings highlight design implications for children’s AI literacy for responsible AI development: supporting proactive, iterative engagement in the development cycle, integrating scaffolds into coherent workflows, and balancing creativity with accuracy.

Tutoring systems improve learning through tailored interventions, such as worked examples, but often suffer from the aptitude-treatment interaction effect where low prior knowledge learners benefit more. We applied the ICAP learning theory to design two new types of worked examples, Buggy (students fix bugs), and Guided (students complete missing rules), requiring varying levels of cognitive engagement, and investigated their impact on learning in a controlled experiment with 155 undergraduate students in a logic problem solving tutor. Students in the Buggy and Guided examples groups performed significantly better on the posttest than those receiving passive worked examples. Buggy problems helped high prior knowledge learners whereas Guided problems helped low prior knowledge learners. Behavior analysis showed that Buggy produced more exploration-revision cycles, while Guided led to more help-seeking and fewer errors. This research contributes to the design of interventions in logic problem solving for varied levels of learner knowledge and a novel application of behavior analysis to compare learner interactions with the tutor.

As artificial intelligence (AI) becomes increasingly embedded in scientific and technical domains, the ability to engage in AI-integrated STEM problem-solving is emerging as a critical skill for the future STEM workforce. Supporting students in this type of problem-solving requires building a strong foundation in computational thinking, particularly through pedagogically effective and technically robust tools. In this paper, we propose augmenting i-Sail, a block-based programming environment designed for AI-integrated STEM problem-solving, with large language model-driven feedback capabilities to facilitate students' problem-solving while reinforcing key computational thinking skills for middle-grade students. We prompt a large language model with structured knowledge about breadth-first search to provide contextualized, adaptive feedback. The LLM helps students connect their problem-solving steps to the high-level structure of the breadth-first search algorithm and apply this understanding to pathfinding. We present a proof-of-concept evaluation that demonstrates the potential of the system to support the development of computational thinking through AI-integrated problem solving in diverse STEM contexts.

The potential of Large Language Models (LLMs) in education is not trivial, but concerns about academic misconduct, misinformation, and overreliance limit their adoption. To address these issues, we introduce MerryQuery, an AI-powered educational assistant using Retrieval-Augmented Generation (RAG), to provide contextually relevant, course-specific responses. MerryQuery features guided dialogues and source citation to ensure trust and improve student learning. Additionally, it enables instructors to monitor student interactions, customize response granularity, and input multimodal materials without compromising data fidelity. By meeting both student and instructor needs, MerryQuery offers a responsible way to integrate LLMs into educational settings.

We propose self-organizing and simplifying neuro-fuzzy networks (NFNs) to yield transparent human-readable policies by exploiting fuzzy information granulation and graph theory. Deriving from social network analysis, we retain only the frequent-yet-discernible (FYD) patterns in NFNs and apply them to reward-based scenarios. The effectiveness of NFNs from FYD patterns is shown in classic control and a real-world classroom using an intelligent tutoring system to teach students.

Students often enroll in university programs to learn the skills necessary to enter the workforce or attend a graduate program. However, students feel an increasing need to supplement their degree programs with online, on-demand (OOD) coursework. Undergraduate computer science (CS) students' motivations for taking OOD CS coursework, how they perceive their usage of OOD CS coursework, and why they stop are not well understood. To address this research gap, we surveyed 51 undergraduate students enrolled in or interested in computer science about their attitudes and usage of OOD CS coursework. We additionally interviewed four of the survey takers to understand the factors that drove them to enroll and stop OOD CS coursework. Our results showed that students perceived OOD CS coursework positively. There is a positive correlation between student self-perceptions of learning during OOD CS coursework and intentions to persist in OOD CS coursework. Students reported primarily using OOD coursework to prepare for interviews and build skills for their resumes. The main reason for attrition was the loss of trial access and replacement by LLMs such as ChatGPT. The contributions of this work include design insights for OOD CS platforms to increase retention and lower attrition, and the understanding that undergraduate CS students perceive free OOD CS coursework as a useful tool to supplement their undergraduate studies.

Block-Based Programming (BBP) platforms, such as Snap!, have become increasingly prominent in $\mathrm{K}-12$ computer science education due to their ability to simplify programming concepts and foster computational thinking from an early age. While these platforms engage students through visual and gamified interfaces, teachers often face challenges in using them effectively and finding all the necessary features for classroom management. To address these challenges, we introduce SnapClass, a classroom management system integrated within the Snap! programming environment. SnapClass was iteratively developed drawing on established research about the pedagogical and logistical challenges teachers encounter in computing classrooms. Specifically, SnapClass allows educators to create and customize block-based coding assignments based on student skill levels, implement rubric-based auto-grading, and access student code history and recovery features. It also supports monitoring student engagement and idle time, and includes a help dashboard with a “raise hand” feature to assist students in real time. This paper describes the design and key features of SnapClass those are developed and those are under progress.

Background and Context. This innovative practice full paper describes the development and implementation of a professional development (PD) opportunity for secondary teachers to learn about ChatGPT. Incorporating generative AI techniques from Large Language Models (LLMs) such as ChatGPT into educational environments offers unprecedented opportunities and challenges. Prior research has highlighted their potential to personalize feedback, assist in lesson planning, generate educational content, and reduce teachers' workload, alongside concerns such as academic integrity and student privacy. However, the rapid adoption of LLMs since ChatGPT's public release in late 2022 has left educators, particularly at the secondary level, with a lack of clear guidance on how LLMs work and can be effectively adopted. Objective. This study aims to introduce a comprehensive, free, and vetted ChatGPT course tailored for secondary teachers, with the objective of enhancing their technological competencies in LLMs and fostering innovative teaching practices. Method. We developed a five-session interactive course on ChatGPT capabilities, limitations, prompt-engineering techniques, ethical considerations, and strategies for incorporating ChatGPT into teaching. We introduced the course to six middle and high school teachers. Our curriculum emphasized active learning through peer discussions, hands-on activities, and project-based learning. We conducted pre- and post-course focus groups to determine the effectiveness of the course and the extent to which teachers' attitudes toward the use of LLMs in schools had changed. To identify trends in knowledge and attitudes, we asked teachers to complete feedback forms at the end of each of the five sessions. We performed a thematic analysis to classify teacher quotes from focus groups' transcripts as positive, negative, and neutral and calculated the ratio of positive to negative comments in the pre- and post-focus groups. We also analyzed their feedback on each individual session. Finally, we interviewed all participants five months after course completion to understand the longer-term impacts of the course. Findings. Our participants unanimously shared that all five of the sessions provided a deeper understanding of ChatGPT, featured enough opportunities for hands-on practice, and achieved their learning objectives. Our thematic analysis underlined that teachers gained a more positive and nuanced understanding of ChatGPT after the course. This change is evidenced quantitatively by the fact that quotes with positive connotations rose from 45% to 68% of the total number of positive and negative quotes. Participants shared that in the longer term, the course improved their professional development, understanding of ChatGPT, and teaching practices. Implications. This research underscores the effectiveness of active learning in professional development settings, particularly for technological innovations in computing like LLMs. Our findings suggest that introducing teachers to LLM tools through active learning can improve their work processes and give them a thorough and accurate understanding of how these tools work. By detailing our process and providing a model for similar initiatives, our work contributes to the broader discourse on teaching professional educators about computing and integrating emerging technologies in educational and professional development settings.

Background and Context. The importance of CS to 21st-century life and work has made it important to find ways to integrate learning CS and programming into the regular school day. However, learning CS is difficult, so teachers integrating programming need effective strategies to scaffold the learning. In this study, we analyze students’ log data and apply a novel technique to compare Parsons Problems with from-scratch programming in a middle school science class. Objectives. Our research questions aimed to investigate whether, how, and when Parsons Problems improve learning efficiency for a programming exercise within science, utilizing log data analysis and an automated progress detector (SPD). Method. We conducted a study on 199 students in a 6th-grade science course, divided into two groups: one engaged with Parsons problems, and the other, a control group, worked on the same programming task without scaffolding. Then, we analyzed differences in performance and coding characteristics between the groups. We also adopted an innovative application of SPD to gain a better understanding of how and when Parsons problems helped students make more progress on the coding task, with an objective measure of final student grades. Findings. The experimental group, with scaffolding through Parsons Problems, achieved significantly higher grades, spent significantly less time programming, and toggled less between block category tabs. Interestingly, they ran their code more frequently compared to the control group. The SPD analysis revealed that the experimental group made significantly higher progress in all four quartiles of their coding time. Implications. Our findings suggest that Parsons problems can improve learning efficiency by enhancing novices’ learning experience without negatively impacting their performance or grades, which is especially important when programming is integrated into K12 courses.

Affective state, referring to an individual's feeling, can impact students' confidence and retention in CS, particularly for novice programmers. However, little research has been conducted to examine how moments that occur during programming impact students' affective states in real-time. In this pilot study, seven undergraduate students in an introductory block-based programming course completed a programming assignment and were surveyed and interviewed about their experience and self-efficacy as programmers. While programming, students periodically recorded their affective states via a popup in the programming environment. We performed retrospective think-aloud interviews with students afterward, asking them to watch and reflect on recordings of their programming. We subsequently analyzed student interviews using thematic analysis to derive 206 codes. These codes were grouped into three areas that impacted affect: the environment, objective progress, and perceptions during programming. To explore why students responded as they did to moment occurrence, we further categorized students based on four dimensions: programming experience, assignment completion, confidence, and the impact of the programming session on self-efficacy. Our initial results suggest that while certain moments elicit similar affective states among students, the interaction of the aforementioned four dimensions may have a higher impact on novices' affective states during programming. We conclude with recommendations for educators to improve students' affective states during and after programming.

Many SIGCSE attendees are either developing or using online educational tools, and all will benefit from better interoperability among these tools and better analysis of the clickstream data coming from those tools. New tools for analyzing big data leveraged by AI (e.g., deep learning for assessment) in turn improve both content and pedagogy, thus setting up a virtuous cycle fueling learning discoveries and leveraging innovation in AI: Online technologies → big data analysis → better online technologies. This NSF-supported workshop is the latest in a series of SPLICE workshops, and is a continuation of our event at SIGCSE 2023, where the SPLICE-Portal, a dedicated socio-technical research infrastructure for Computing Education Research, was presented. This year, we continue the work with several new SPLICE community working groups, including those on Dashboards, Large Language Models, Parsons Problems, and Smart Learning Content Protocols. We continue to build upon our existing collaborations developed over the course of the project to engage more members of the community in tasks that will advance the project agenda.

In the field of computer science, cultural assumptions are embedded in programming languages and problem prompts. However, for students studying computer science in a Western country not from a Western culture, these assumptions can create learning barriers. This paper investigates the impact of cultural assumptions on international students studying computer science in a Western country; with a focus on understanding the barriers students face, how they overcome these barriers, and how barriers can be avoided. By performing thematic analysis on semi-structured interviews with 12 international graduate students at North Carolina State University, the authors found six main themes: Barriers, Increased Work, Emotions, Educational Environment, Overcoming Barriers, and Solutions to Barriers. Analyzing these themes provided insight into what barriers international students face and how they can be alleviated. The most common suggestions participants gave for alleviating barriers were more discussions around problem statements. By shedding light on this topic, the authors hope to inform computer science educators and researchers on the importance of creating inclusive and culturally relevant learning environments that accommodate the needs of diverse student populations.

In computing classrooms, building an open-ended programming project engages students in the process of designing and implementing an idea of their own choice. An explicit planning process has been shown to help students build more complex and ambitious open-ended projects. However, novices encounter difficulties in exploring and creatively expressing ideas during planning. We present Idea Builder, a storyboarding-based planning system to help novices visually express their ideas. Idea Builder includes three features: 1) storyboards to help students express a variety of ideas that map easily to programming code, 2) animated example mechanics with example actors to help students explore the space of possible ideas supported by the programming environments, and 3) synthesized starter code to help students easily transition from planning to programming. Through two studies with high school coding workshops, we found that students self-reported as feeling creative and feeling easy to communicate ideas; having access to animated example mechanics of an actor help students to build those actors in their plans and projects; and that most students perceived the synthesized starter code from Idea Builder as helpful and time-saving.

Multi-pronged programs that involve students in a combination of proven interventions (i.e., tutoring other students, building community, developing skills, etc.) constitute one pedagogical approach to increasing the number and diversity of computing professionals. In this manuscript, we evaluate the efficacy of one such multi-pronged program, the STARS Computing Corps, a Broadening Participation in Computing Alliance program funded by the National Science Foundation. These analyses improve upon previous efforts to assess the efficacy of STARS by examining dosage effects of the program, adding controls for students' initial intentions to pursue computing, and conducting these analyses at various points in a student's participation in STARS. We also conduct analyses to determine the efficacy of various STARS activities. Controlling for students' initial intentions to persist in computing, we find robust evidence that spending more time each week on STARS' activities positively predicts students' intentions to persist in a computing career, and that STARS has a heightened positive impact on Black and Hispanic students. We do not find evidence that the number of semesters a student spends in STARS is predictive of computing persistence, nor do we find differences in the efficacy of various STARS activities. In sum, these results suggest that STARS has a positive impact on students' intentions to persist in computing and that multi-pronged programs like STARS should focus on the intensity of participation (as opposed to the length of participation or a particular activity) to increase students' desire to persist in computing careers.

Many students struggle with decomposition and planning despite the necessity of these skills in computing education. Hence, more tools are needed to scaffold these processes. In this paper, we present Jigsaw, a standalone visual planning tool to help students practice decomposition and planning before writing code. Jigsaw allows students to compose a solution to a new problem based on previously seen “patterns,” such as the accumulator pattern for summing values or the filter pattern for conditional input selection. Students can connect these patterns together to see how data flows between them and define a solution plan. Jigsaw’s goal is to scaffold students’ planning processes by presenting relevant patterns for a given problem. Using a within-subjects design, we evaluated Jigsaw by observing 17 undergraduate students as they planned for and implemented two programming assignments. The experimental task included Jigsaw, and the control task did not. This design aimed to understand how the tool impacted students’ planning and programming process. Subsequently, we conducted interviews with these students regarding their planning and programming experiences with and without Jigsaw. Many students explicitly mentioned they would employ Jigsaw for planning and appreciated the scaffolding it provided. Students also admired the Jigsaw’s novelty in visualizing programming problems. We conclude with our design takeaways and recommendations for future work.

The assistance dilemma is a well-recognized challenge to determine when and how to provide help during problem solving in intelligent tutoring systems. This dilemma is particularly challenging to address in domains such as logic proofs, where problems can be solved in a variety of ways. In this study, we investigate two data-driven techniques to address the when and how of the assistance dilemma, combining a model that predicts when students need help learning efficient strategies, and hints that suggest what subgoal to achieve. We conduct a study assessing the impact of the new pedagogical policy against a control policy without these adaptive components. We found empirical evidence which suggests that showing subgoals in training problems upon predictions of the model helped the students who needed it most and improved test performance when compared to their control peers. Our key findings include significantly fewer steps in posttest problem solutions for students with low prior proficiency and significantly reduced help avoidance for all students in training.

Motivation. We have created a modular project-based learning curriculum, Computer Science Frontiers (CSF) [1, 8], for secondary students in attempts to increase the persistence of computer science (CS) students in higher education. The CSF course is divided into four different modules (Distributed Computing, Internet of Things, Artificial Intelligence, and Software Engineering), each centered around a topic typically introduced to students only in higher education. Using the block-based programming environment NetsBlox [4], students are able to access various Application Programming Interfaces related to their interests [2, 3]. The goal of this course is to increase student interest in CS during high school - when first career choices occur [7] - in hopes they will persist in CS during their undergraduate studies.

Background and Context: Open-ended programming projects encourage novice students to choose and pursue projects based on their own ideas and interests, and are widely used in many introductory programming courses. However, novice programmers encounter challenges exploring and discovering new ideas, implementing their ideas, and applying unfamiliar programming concepts and APIs. Code examples are one of the primary resources students use to apply code usage patterns and learn API knowledge, but little work has investigated the effect of having access to examples on students’ open-ended programming experience.

Many students rely on examples when learning to program, but they often face barriers when incorporating these examples into their own code and learning the concepts they present. As a step towards designing effective example interfaces that can support student learning, we investigate novices' needs and strategies when using examples to write code. We conducted a study with 12 pairs of high school students working on open-ended game design projects, using a system that allows students to browse examples based on their functionality, and to view and copy the example code. We analyzed interviews, screen recordings, and log data, identifying 5 moments when novices request examples, and 4 strategies that arise when students use examples. We synthesize these findings into principles that can inform the design of future example systems to better support students.

We explore eXplainable AI (XAI) to enhance user experience and understand the value of explanations in AI-driven pedagogical decisions within an Intelligent Pedagogical Agent (IPA). Our real-time and personalized explanations cater to students' attitudes to promote learning. In our empirical study, we evaluate the effectiveness of personalized explanations by comparing three versions of the IPA: (1) personalized explanations and suggestions, (2) suggestions but no explanations, and (3) no suggestions. Our results show the IPA with personalized explanations significantly improves students' learning outcomes compared to the other versions.

A key goal of many computer science education efforts is to increase the number and diversity of students who persist in the field of computer science and into computing careers. Many interventions have been developed in computer science designed to increase students' persistence in computing. However, it is often difficult to measure the efficacy of such interventions, as measuring actual persistence by tracking student enrollments and career placements after an intervention is difficult and time-consuming, and sometimes even impossible. In the social sciences, attitudinal research is often used to solve this problem, as attitudes can be collected in survey form around the same time that interventions are introduced and are predictive of behavior. This can allow researchers to assess the potential efficacy of an intervention before devoting the time and energy to conduct a longitudinal analysis. In this paper, we develop and validate a scale to measure intentions to persist in computing, and demonstrate its use in predicting actual persistence as defined by enrolling in another computer science course within two semesters. We conduct two analyses to do this: First, we develop a computing persistence index and test whether our scale has high alpha reliability and whether our scale predicts actual persistence in computing using students' course enrollments. Second, we conduct analyses to reduce the number of items in the scale, to make the scale easy for others to include in their own research. This paper contributes to research on computing education by developing and validating a novel measure of intentions to persist in computing, which can be used by computer science educators to evaluate potential interventions. This paper also creates a short version of the index, to ease implementation.

The current CS for All movement reflects a need to provide opportunities for every person to learn computer science, as a matter of equity. Many computing professionals have a desire to contribute to society; since access to computing education is limited in many ways, finding ways to use computing to give back is particularly valuable. In this Birds of a Feather, we will discuss strategies for individual students, faculty, and institutions to develop and lead efforts focused on improving inclusion, equity, and diversity in computing. Exemplary efforts that computing students, faculty, and computing professionals have used to broaden participation include mentoring, tutoring, K12 outreach, expanded research opportunities, and service learning. In this BoF, we invite participants to seek and share resources that can lower the barriers to engaging in outreach and research for broadening participation in computing. We will also discuss strategies for choosing projects that leverage local resources and opportunities, promote personal professional development, and contribute to institutional improvement.

One of SIGCSE's missions is to provide occasion for K-12 teachers and college/university professors to network, developing understanding of each other's roles as educators and generating ideas for collaborations for the sake of broadening participation in computing. We aim to promote opportunities for researchers and K-12 practitioners to work together in a BoF in order to participate in discussion regarding a)K-12 members' interests regarding CS education research; b) strategies for increasing opportunities for K-12 membersc) researcher/K12 collaborations d) ideas for future grants and/or collaborations The expected audience are the college/university faculty interested in K-12 education, as well as K-12 teachers or administrators interested in college/university CS education research. Regarding K-12/University collaborations, one might only think of the benefits to students. Yet, opportunities for learning and growth on the part of teachers and university faculty as work partners fosters motivation on both sides. This unexpected, valuable outcome has been documented in the literature as well [1,2,3]. The organizers of this BoF each have valued the learning from such collaborations and represent various roles as collaborative researchers from both sides.

As computer science is increasingly taught in secondary schools, tools need to integrate block-based environments into learning platforms. This way, teachers can more effectively lead lessons, help students, and assess students' programs in their classrooms. We conducted a participatory design process with three K-12 computing teachers to understand their struggle and needs for block coding within their classrooms. The teachers identified 14 needs that were not already addressed by our tool, SnapClass. SnapClass, a new web-based learning platform for Snap!, integrates assignments with starter code, executable student submissions, rubric-based assessment, and a gradebook into one platform. The teachers designed prototypes for three features important to their classrooms: assignment differentiation, help-requests, and peer and self-assessment. This paper begins by introducing SnapClass and the motivation for its development. Then through thematic analysis of the session transcripts, we identify the common struggles teachers face while instructing programming and summarize how they would address those struggles through the design of SnapClass.

Many students struggle when they are first learning to program. Without help, these students can lose confidence and negatively assess their programming ability, which can ultimately lead to dropouts. However, detecting the exact moment of student struggle is still an open question in computing education. In this work, we conducted a think-aloud study with five high-school students to investigate the automatic detection of progressing and struggling moments using a detector algorithm (SPD). SPD classifies student trace logs into moments of struggle and progress based on their similarity to prior students' correct solutions. We explored the extent to which the SPD-identified moments of struggle aligned with expert-identified moments based on novices' verbalized thoughts and programming actions. Our analysis results suggest that SPD can catch students' struggling and progressing moments with a 72.5% F1-score, but room remains for improvement in detecting struggle. Moreover, we conducted an in-depth examination to discover why discrepancies arose between expert-identified and detector-identified struggle moments. We conclude with recommendations for future data-driven struggle detection systems.

Historically, female students have shown low interest in the field of computer science. Previous computer science curricula have failed to address the lack of female-centered computer science activities, such as socially relevant and real-life applications. Our new summer camp curriculum introduces the topics of artificial intelligence (AI), machine learning (ML) and other real-world subjects to engage high school girls in computing by connecting lessons to relevant and cutting edge technologies. Topics range from social media bots, sentiment of natural language in different media, and the role of AI in criminal justice, and focus on programming activities in the NetsBlox and Python programming languages. Summer camp teachers were prepared in a week-long pedagogy and peer-teaching centered professional development program where they concurrently learned and practiced teaching the curriculum to one another. Then, pairs of teachers led students in learning through hands-on AI and ML activities in a half-day, two-week summer camp. In this paper, we discuss the curriculum development and implementation, as well as survey feedback from both teachers and students.

Student modeling sits at the epicenter of adaptive learning technology. In contrast to the voluminous work on student modeling for well-defined domains such as algebra, there has been little research on student modeling in programming (SMP) due to data scarcity caused by the unbounded solution spaces of open-ended programming exercises. In this work, we focus on two essential SMP tasks: program classification and early prediction of student success and propose a Cross-Lingual Adversarial Domain Adaptation (CrossLing) framework that can leverage a large programming dataset to learn features that can improve SMP's build using a much smaller dataset in a different programming language. Our framework maintains one globally invariant latent representation across both datasets via an adversarial learning process, as well as allocating domain-specific models for each dataset to extract local latent representations that cannot and should not be united. By separating globally-shared representations from domain-specific representations, our framework outperforms existing state-of-the-art methods for both SMP tasks.

Are women less likely to persist in computer science because of gender differences in self-assessed computing ability? And why do gender differences exist in self-assessments among women and men who earn the same grades? We use a mixed-method research design to answer these questions, utilizing both quantitative survey data (n = 764) and qualitative interview data (n = 59) from students in introductory computing courses at a large U.S. state university. Quantitatively, we find that women self-assess their computing ability significantly lower than men who earn the same grades, and that these lower self-assessments reduce the likelihood that women enroll in future CS courses (relative to men who earn equivalent grades). Qualitatively, we explore how women and men perceive their own computing ability to understand why women self-assess their ability lower than men. Our interviews revealed that women were much less likely than men to make favorable comparative judgements about their ability relative to their classmates. Women also had higher personal performance standards than men. Lastly, women were more likely than men to experience disrespectful treatment, with an undertone of presumed incompetence, from their TAs and classmates. In sum, this research furthers our understanding of why gender differences exist in self-assessments of computing ability and how these differences can contribute to gender disparities in computing persistence. It also draws attention to the importance of feedback in computing courses and suggests that improving course feedback may reduce gender disparities in computing.

Research has shown that high self-assessment of ability, sense of belonging, and professional role confidence are crucial for students' persistence in computing. As grades in introductory computer science courses tend to be lower than other courses, it is essential to provide students with contextualized feedback about their performance in these courses. Giving students unambiguous and con- textualized feedback is especially important during COVID when many classes have moved online and instructors and students have fewer opportunities to interact. In this study, we investigate the effect of a lightweight, scalable intervention where students received personalized, contextualized feedback from their instructors after two major assignments during the semester. After each intervention, we collected survey data to assess students' self-assessment of computing ability, sense of belonging, intentions to persist in computing, professional role confidence, and the likelihood of stating intention to pursue a major in computer science. To analyze the effectiveness of our intervention, we conducted linear regression and mediation analysis on student survey responses. Our results have shown that providing students with personalized feedback can significantly improve their self-assessment of computing ability, which will significantly improve their intentions to persist in computing. Furthermore, our results have demonstrated that our intervention can significantly improve students' sense of belonging, professional role confidence, and the likelihood of stating an intention to pursue a major in computer science.

Our researchers seek to support students in building block-based programming projects that are motivating and engaging as well as valuable practice in learning to code. A difficult part of the programming process is planning. In this research, we explore how novice programmers used a custom-built planning tool, PlanIT, contrasted against how they used storyboarding when planning games. In a three-part study, we engaged novices in planning and programming three games: a maze game, a break-out game, and a mashup of the two. In a set of five case studies, we show how five pairs of students approached the planning and programming of these three games, illustrating that students felt more creative when storyboarding rather than using PlanIT. We end with a discussion on the implications of this work for designing supports for novices to plan open-ended projects.

The Beauty and Joy of Computing (BJC) is a CS Principles (CSP) course created at UC Berkeley to reach high school and university nonmajors in computer science. It was chosen for the CSP pilot and endorsed by the College Board as an AP CSP curriculum and PD provider since the first AP CSP exam sitting in 2017. This past year, BJC developed a new course for middle school and early high school that teaches a functional approach to programming, emphasizing iteration and commands, and including exciting projects in graphics, data, and media. In this workshop, we will provide an overview of the BJC middle school and high school curriculum including course materials, teacher resources, and an introduction to Snap!, the chosen visual programming language of BJC. We will not only cover curriculum updates, Snap! updates, BJC for middle school, BJC in other languages, but also dive into labs with hands-on programming. Laptop is required.

The Computer Science Frontiers (CSF) project introduces teachers to the topics of artificial intelligence and distributed computing to engage their female students in computing by connecting lessons to relevant cutting edge technologies. Application topics include social media and news articles, as well as climate change, the arts (movies, music, and museum collections), and public health/medicine. CSF educators are prepared in a pedagogy and peer-teaching centered professional development program where they simultaneously learn and teach distributed computing, artificial intelligence, and internet of things lessons to each other. These professional developments allow educators to hone in on their teaching skills of these new topics and gain confidence in their ability to teach new computer science materials before running several activities with their students in the academic year classroom. In this workshop, teachers participating in the CS Frontiers professional development will give testimonials discussing their experiences teaching these topics in a two week summer camp. Attendees will then try out three computing activities, one from each Computer Science Frontiers module. Finally, there will be a question and answer session.

We have found that giving top-performing students in CS1 courses personalized feedback increases their intentions to persist in computing, especially among students who are women. This personalized feedback also appears to improve students' course experience and increases the likelihood that women apply to be CS1 TAs. Yet despite these benefits, giving personalized feedback may seem too impractical and time-intensive for faculty members to adopt in their own classrooms. In this workshop, we will reduce the burden of giving students personalized feedback by: 1) giving instructors empirically validated email templates to use in their own courses, and 2) guiding faculty how to send emails at-scale. We will also discuss how self-assessments influence students' career choices, how gender stereotypes bias self-assessments, and what faculty can do to counteract biased self-assessments of computing ability.

Classroom dashboards are designed to help instructors effectively orchestrate classrooms by providing summary statistics, activity tracking, and other information. Existing dashboards are generally specific to an LMS or platform and they generally summarize individual work, not group behaviors. However, CS courses typically involve constellations of tools and mix on- and offline collaboration. Thus, cross-platform monitoring of individuals and teams is important to develop a full picture of the class. In this work, we describe our work on Concert, a data integration platform that collects data about student activities from several sources such as Piazza, My Digital Hand, and GitHub and uses it to support classroom monitoring through analysis and visualizations. We discuss team visualizations that we have developed to support effective group management and to help instructors identify teams in need of intervention.

Academic computing departments are seeking ways to broaden participation in computing (BPC), and many are encouraging individual faculty, staff, and students to attend diversity-oriented conferences, like the Tapia and STARS Celebrations of Diversity in Computing and Grace Hopper Celebration of Women in Computing. The purpose of the STARS Ignite Workshop is to provide faculty and staff with a framework including the tools and knowledge needed to make such conference attendance beneficial both to the students attending the conference and to the sponsoring faculty and their department. In this workshop, participants will learn to recruit and lead a BPC purpose-driven student conference cohort and the implementation of a BPC event or program. Participants will be provided with opportunities to adapt sample materials (recruitment emails, applications, etc.) to their own needs, assess the BPC needs at their institutions, and learn how to determine if their BPC efforts are successful. A laptop and Google account are required for this event. Interested attendees should be willing to commit to leading a student conference cohort to a diversity-oriented computing conference and then implementing a BPC event or program with these students.

Open-ended programming engages students by connecting computing with their real-world experience and personal interest. However, such open-ended programming tasks can be challenging, as they require students to implement features that they may be unfamiliar with. Code examples help students to generate ideas and implement program features, but students also encounter many learning barriers when using them. We explore how to design code examples to support novices' effective example use by presenting our experience of building and deploying Example Helper, a system that supports students with a gallery of code examples during open-ended programming. We deployed Example Helper in an undergraduate CS0 classroom to investigate students' example usage experience, finding that students used different strategies to browse, understand, experiment with, and integrate code examples, and that students who make more sophisticated plans also used more examples in their projects.

Block-based programming environments, such as Scratch and Snap!, engage users to create programming artifacts such as games and stories, and share them in an online community. Many Snap! users start programming by reusing and modifying an example project, but encounter many barriers when searching and identifying the relevant parts of the program to learn and reuse. We present Pinpoint, a system that helps Snap! programmers understand and reuse an existing program by isolating the code responsible for specific events during program execution. Specifically, a user can record an execution of the program (including user inputs and graphical output), replay the output, and select a specific time interval where the event of interest occurred, to view code that is relevant to this event. We conducted a small-scale user study to compare users’ program comprehension experience with and without Pinpoint, and found suggestive evidence that Pinpoint helps users understand and reuse a complex program more efficiently.

In recent years, research has increasingly focused on developing intelligent tutoring systems that provide data-driven support for students in need of assistance during programming assignments. One goal of such intelligent tutors is to provide students with quality interventions comparable to those human tutors would give. While most studies focused on generating different forms of on-demand support, such as next-step hints and worked examples, at any given moment during the programming assignment, there is a lack of research on why human tutors would provide different forms of proactive interventions to students in different situations. This information is critical to know to allow the intelligent programming environments to select the appropriate type of student support at the right moment.

This article examines the grading process and profiles of secondary computer science teachers as they assess block-based student programming submissions. Through an iterative design process, we have created a new tool, Gradesnap, which streamlines how teachers can open, review, and evaluate student submissions within the same interface. Our study compares teachers’ grading processes using the different assessment formats, so that we can understand how their grading processes can be augmented or supported to reduce ’pain points’ and to enable teachers to provide more constructive and formative feedback for students. We use a case study approach to examine the experiences and outcomes of four secondary computer science teachers with varied teaching and assessment experience, when grading as usual, grading with a rubric, and grading with GradeSnap. Our study shows that when participants use GradeSnap, they are able to give supportive comments to lower performing and borderline students who need critical feedback to better understand misconceptions. We also discovered that the different grading processes provided a vehicle for reflection for some teachers in understanding their grading goals and how they enact them. This research is the first to examine teacher grading processes for computer science, and highlights the need for teacher preparation and support for providing programming feedback and assessment.

We compare subject area block-based programming lessons made by two participant groups new to creating K-12 computing curricula. The first group consists of 29 high school interns with prior programming experience but not formal pedagogical training. The second group consists of 86 teachers who attended an infusing computing professional development, with formal pedagogical training but no background in programming. We examine a total of 113 lessons for their use of scaffolding, teacher accessibility, equity, and computing and subject area content. Our analysis extends prior work showing novice-student creator strength in conveying content knowledge and difficulty in incorporating equity into coding lessons. Compared to lessons created by the teacher group (less coding, more pedagogical experience), our results show that teachers also struggle with including equitable practices and opportunity for culturally responsive and identify affirming activities. However, they were much stronger in including assessment focused items. Furthermore, teachers were able to provide adequate support for coding and content knowledge. Our exploration of scaffolding included in these lessons shows that scaffolding included differs by creator and that projects involving worked examples were more often correlated to more evidence of equitable practices. Through this research, we gain insight into what additional knowledge and training are needed to help creators make higher quality and more accessible computing lessons for non-computer science courses.

Open-ended programming increases students' motivation by allowing them to solve authentic problems and connect programming to their own interests. However, such open-ended projects are also challenging, as they often encourage students to explore new programming features and attempt tasks that they have not learned before. Code examples are effective learning materials for students and are well-suited to supporting open-ended programming. However, there is little work to understand how novices learn with examples during open-ended programming, and few real-world deployments of such tools. In this paper, we explore novices' learning barriers when interacting with code examples during open-ended programming. We deployed Example Helper, a tool that offers galleries of code examples to search and use, with 44 novice students in an introductory programming classroom, working on an open-ended project in Snap. We found three high-level barriers that novices encountered when using examples: decision, search, and integration barriers. We discuss how these barriers arise and design opportunities to address them.

This panel is an outgrowth of a Technology that Educators of Computing Hail (TECH) Birds of a Feather session held at SIGCSE for seven years, which has grown into popular panels for four years, and served as a springboard for a regular column in ACM Inroads. It will provide a chance for seasoned high school and university educators to show the technologies that they can't live without, what problems they solve, and how to use them.

The Beauty and Joy of Computing (BJC) is a CS Principles (CSP) course created at UC Berkeley to reach high school and university non-majors in computer science. It was chosen for the CSP pilot and endorsed by the College Board as an AP CSP curriculum and PD provider since the first AP CSP exam sitting in 2017. In this workshop, we will provide an overview of the BJC curriculum including course materials, teacher resources, and an introduction to Snap!, the chosen visual programming language of BJC. We will not only cover curriculum updates, Snap! updates, BJC for middle school, BJC in other languages, but also dive into labs with hands-on programming. Some of our experienced BJC Lead Teachers will provide support and answer questions based on personal classroom experience. This workshop is intended to be an introduction and update to the Beauty and Joy of Computing curriculum (bjc.edc.org) and community. This is not intended to be an in-depth experience, but an overview. Note: It is recommended to use least a Chromebook during the workshop in order to achieve the learning outcomes. Also, having Multiple/dual monitors will make for a better experience.

Computational modeling provides an excellent vehicle for raising scientific awareness of emergent and topical phenomena such as COVID-19. Now more than ever, it is crucial to provide students with factual information about how diseases spread and how their own actions can impact that spread. In order to both encourage computational thinking skills and build scientific knowledge of the COVID-19 pandemic, we have created a series of programming activities through which students construct their own computational models based on the emerging scientific consensus around COVID-19. Students are able to model everyday situations such as being in a crowded area or going to stores while unknowingly infected, and immediately see the consequences of those actions. By including accurate scientific variables such as the reproductive number of the virus, incubation period, and period of communicability, students are able to create their own epi-curves that demonstrate the severity of the disease and provide students with visual representation of how quickly COVID-19 spreads. We also use the scientific model and associated modeling activities to reinforce best practices at home and in the community. Finally, this curriculum development effort demonstrates how block-based computational modeling activities lend themselves to agile curricular re-design around emerging and topics of local interest

Academic computing departments are seeking ways to broaden participation, and many are encouraging individual faculty, staff, and students to attend diversity-oriented conferences, like the Tapia and STARS Celebrations of Diversity in Computing and Grace Hopper Celebration of Women in Computing. Such conferences present opportunities to meet a broader community of people for professional development and networking, to be inspired by leaders in computing, and to celebrate diversity. However, while many institutions sponsor these conferences and support individual student attendance, students may not know how to leverage these opportunities effectively. We argue that leading a cohort of faculty/staff and students who attend a conference with the shared goal of broadening participation can provide lasting benefits for computing departments. This workshop will prepare faculty and staff to recruit and lead a team of students and leverage conference attendance to ignite broadening participation efforts. Through a hands-on collaborative process, the workshop provides the knowledge and tools needed to successfully lead a cohort, and helps attendees tailor the provided tools to their local strengths and needs to broaden participation in computing.

In 2018 and 2019, Infusing Computing offered face-to-face summer PD workshops to support middle and high school teachers in integrating computational thinking into their classrooms through week-long summer PD workshops and academic-year support. Due to COVID-19, 151 teachers attended the Summer 2020 PD workshops in a week-long virtual conference format. In this paper, we describe Virtual Pivot: Infusing Computing, which employed emerging technology tools, pre-PD training, synchronous and asynchronous sessions, Snap! pair programming, live support, and live networking. Drawing on findings from participant interviews and post-PD surveys, we argue that three categories of changes (digital tools, formats, and supports for teacher engagement and collaboration) were effective in increasing participants' self-efficacy in teaching CT, supporting collaboration, and enabling participants to design CT-infused content-area lessons. We conclude by discussing how elements of this virtual PD can be replicated to increase teacher and student access to CT practices in middle and high school classrooms

Project-based learning can encourage and motivate students to learn through exploring their own interests, but introduces special challenges for novice programmers. Recent research has shown that novice students perceive themselves to be "bad at programming, especially when they do not know how to start writing a program, or need to create a plan before getting started. In this paper, we present PlanIT, a guided planning tool integrated with the Snap! programming environment designed to help novices plan and program their open-ended projects. Within PlanIT, students can add a description for their project, use a to do list to help break down the steps of implementation, plan important elements of their program including actors, variables, and events, and view related example projects. We report findings from a pilot study of high school students using PlanIT, showing that students who used the tool learned to make more specific and actionable plans. Results from student interviews show they appreciate the guidance that PlanIT provides, as well as the affordances it offers to more quickly create program elements.

We have established a research-practice partnership (RPP) to build a computer science (CS) and computational thinking (CT)-focused STEM ecosystem at two middle schools. Creating such an ecosystem to broaden student participation in computing through an RPP approach involves all stakeholders in the research process. Borrowing upon visual participatory research methods, we developed a graphic research instrument to engage teachers in the research process and elicit their perspectives on strategies for building the ecosystem. This experience report describes our research methodology across two distinct cases to demonstrate the utility of this drawing activity as an investigative and partnership development tool. The contribution is in offering a flexible approach to other university-based RPP teams that enables a synergistic partnership development tool and data collection instrument that can be tailored to a variety of RPP contexts, facilitating more productive and equitable ways of engaging stakeholders in the research process. We describe our project contexts and share results from the pilot study with practitioner-members of our RPP teams. We discuss two cases to highlight the contribution this approach made to the development of our partnerships.

Analytics about how students navigate online learning tools throughout the duration of an assignment is scarce. Knowledge about how students use online tools before a course's end could positively impact students' learning outcomes. We introduce PEDI (Piazza Explorer Dashboard for Intervention), a tool which analyzes and presents visualizations of forum activity on Piazza, a question and answer forum, to instructors. We outline the design principles and data-informed recommendations used to design PEDI. Our prior research revealed two critical periods in students' forum engagement over the duration of an assignment. Early engagement in the first half of an assignment duration positively correlates with class average performance. Whereas, extremely high engagement toward the deadline predicted lower class average performance. PEDI uses these findings to detect and flag troubling engagement levels and informs instructors through clear visualizations to promote data-informed interventions. By providing insights to instructors, PEDI may improve class performance and pave the way for a new generation of online tools.

Many block-based programming environments have proven to be effective at engaging novices in learning programming. However, most restrict access to the outside world, limiting learners to commands and computing resources built in to the environment. Some allow learners to drag and drop files, connect to sensors and robots locally or issue HTTP requests. But in a world where most of the applications in our daily lives are distributed (i.e., their functionality depends on communicating with other programs or accessing resources and data on the internet), the lack of support for beginners to envision and create such distributed programs is a lost opportunity. This paper argues that it is not only feasible, but crucial, to create environments with simple yet powerful abstractions that open up distributed computing and other widely used but advanced computing concepts including networking, the Internet of Things, and cybersecurity to novices. By thus removing the walls around our environments, we can expand opportunities for learning considerably: programs can access a wealth of online data and web services, and communicate with other projects. Moreover, these changes can enable young learners to collaborate with each other during program construction whether they share their physical location or study remotely. Importantly, providing access to the wider world will also help counter widespread student perceptions that block-based environments are mere toys, and show that they are capable of creating compelling applications. The paper presents NetsBlox, a programming environment that supports these ideas and shows that tools can be designed to democratize access to powerful ideas in computing.

Prior work suggests that novice programmers are greatly impacted by the feedback provided by their programming environments. While some research has examined the impact of feedback on student learning in programming, there is no work (to our knowledge) that examines the impact of adaptive immediate feedback within programming environments on students' desire to persist in computer science (CS). In this paper, we integrate an adaptive immediate feedback (AIF) system into a block-based programming environment. Our AIF system is novel because it provides personalized positive and corrective feedback to students in real time as they work. In a controlled pilot study with novice high-school programmers, we show that our AIF system significantly increased students' intentions to persist in CS, and that students using AIF had greater engagement (as measured by their lower idle time) compared to students in the control condition. Further, we found evidence that the AIF system may improve student learning, as measured by student performance in a subsequent task without AIF. In interviews, students found the system fun and helpful, and reported feeling more focused and engaged. We hope this paper spurs more research on adaptive immediate feedback and the impact of programming environments on students' intentions to persist in CS.

In interactive e-learning environments such as Intelligent Tutoring Systems, there are pedagogical decisions to make at two main levels of granularity: whole problems and single steps. In recent years, there is growing interest in applying data-driven techniques for adaptive decision making that can dynamically tailor students' learning experiences. Most existing data-driven approaches, however, treat these pedagogical decisions equally, or independently, disregarding the long-term impact that tutor decisions may have across these two levels of granularity. In this paper, we propose and apply an offline Gaussian Processes based Hierarchical Reinforcement Learning (HRL) framework to induce a hierarchical pedagogical policy that makes decisions at both problem and step levels. An empirical classroom study shows that the HRL policy is significantly more effective than a Deep Q-Network (DQN) induced policy and a random yet reasonable baseline policy.

In this paper, we perform a predictive analysis of a curriculum-integrated math game, ST Math, to suggest a partial ordering for the game's curriculum sequence. We analyzed the sequence of ST Math objectives played by elementary school students in 5 U.S. districts and grouped each objective into difficult and easy categories according to how many retries were needed for students to master an objective. We observed that retries on some objectives were high in one district and low in another district where the objectives are played in a different order. Motivated by this observation, we investigated what makes an effective curriculum sequence. To infer a new partially-ordered sequence, we performed an expanded replication study of a novel predictive analysis by a prior study to find predictive relationships between 15 objectives played in different sequences by 3,328 students from 5 districts. Based on the predictive abilities of objectives in these districts, we found 17 suggested objective orderings. After deriving these orderings, we confirmed the validity of the order by evaluating the impact of the suggested sequence on changes in rates of retries and corresponding performance. We observed that when the objectives were played in the suggested sequence, we record a drastic reduction in retries, implying that these objectives are easier for students. This indicates that objectives that come earlier can provide prerequisite knowledge for later objectives. We believe that data-informed sequences, such as the ones we suggest, may improve efficiency of instruction and increase content learning and performance.

We present a data-driven analysis that provides generalized insights of how a curriculum integrated educational math game gets used as a routinized classroom activity throughout the year in authentic primary school classrooms. Our study relates observations from a field study on Spatial Temporal Math (ST Math) to our findings mined from ST Math students' sequential game play data. We identified features that vary across game play sessions and modeled their relationship with session performance. We also derived data-informed suggestions that may provide teachers with insights into how to design classroom game play sessions to facilitate more effective learning.

The Beauty and Joy of Computing (BJC) is a CS Principles course developed at UC Berkeley for high school juniors through university non-majors. Together, UC Berkeley, the Education Development Center, and NC State have brought BJC to 700+ teachers nationwide. Since 2011, NC State has developed regional partnerships and a train-the-trainer model to offer nationwide PD to 600+ high school teachers. Our guiding philosophy is to meet students where they are, but not leave them there. BJC covers the big ideas and computational thinking practices in the AP CSP curriculum framework using Snap!, an easy-to-learn blocks-based programming language, and powerful computing ideas like recursion, higher-order functions, and computability. Through BJC, students create beautiful images, and realize that code itself can be beautiful. Having fun is an explicit course goal. BJC takes a "lab-centric" approach, and most learning occurs through guided programming labs where students explore and play. In this workshop, we will provide an overview of BJC, share experiences as instructors at university and high school levels, and share details of summer PD opportunities. Laptop needed.

This project is a Research to Practice Partnership (RPP) between two middle schools and two universities. It focuses on investigating problems and on identifying solutions around increasing participation and interest in computer science (CS). We aim to do this by identifying, experimenting with, and fine-tuning methods to help students develop computational thinking (CT) skills. The research employs a STEM ecosystem model, which facilitates a support structure that aims to mitigate barriers and impact students as they progress in STEM areas. While this RPP is still a work in progress, we present data from the first year of our collaboration with one of the middle schools. While the research questions guiding this RPP are intended to be iterative and revised annually, year one data provides perspectives on (1) barriers to developing a STEM ecosystem that supports CS/CT for every student through integration into science, math, and language arts courses, (2) the factors or interventions needed for the development of a CS/CT focused ecosystem that supports everyone in the school, (3) the indicators of success for a CS/CT focused STEM ecosystem in a school, and (4) how the ecosystem prepares and engages all students for CS/CT work in high school. Year one data is discussed in terms of the STEM ecosystem framework and in how it will guide the next steps in this partnership. This project contributes to the understanding of how to prepare future generations for participation in a workforce where knowledge of the foundations of CS/CT is integral to success.

While AP Computer Science Principles (CSP) has made great strides in increasing the number of women and underrepresented minorities (URM) taking an AP Computer Science course, the percentage of women and URM passing the AP CSP Exam remains lower than that of non-minority males. The AP CSP Exam consists of two performance-based tasks, create and explore, in addition to a multiple-choice section. We conducted one-on-one interviews with Beauty and Joy of Computing (BJC) Master Teachers to find what they like about the exam, what they do not like, and potential barriers to passing. The teachers mentioned preparedness, including test taking skills, and motivation, specifically to complete the tasks, as reasons for their students' performance on the AP Exam. Community mindset was mentioned as a possible reason for lack of motivation and engagement. Some potential barriers to passing, expressed by teachers, are access to technology, job obligations, and languages in which the AP Exam is offered. Although the create task was well liked by the teachers, some thought the scoring could be improved. In addition, teachers detail how their own assessments compare to the AP CSP Exam, with the majority using programming projects as their main way to assess for student understanding. Half of the teachers cited the amount of time for performance tasks and multiple-choice questions as components of the exam they would change. These findings provide insight into AP CSP Exam performance, teacher suggestions for exam improvements, and details on CS classroom assessments.

Despite the increasing attention to infusing CT into middle and high school content area classrooms, there is a lack of information about the most effective practices and models to support teachers in their efforts to integrate disciplinary content and CT principles. To address this need, this paper proposes the Code, Connect and Create (3C) professional development (PD) model, which was designed to support middle and high school content area teachers in infusing computational thinking into their classrooms. To evaluate the model, we analyzed quantitative and qualitative data collected from Infusing Computing PD workshops designed for in-service science, math, English language arts, and social studies teachers located in two Southeastern states. Drawing on findings from our analysis of teacher-created learning segments, surveys, and interviews, we argue that the 3C professional development model supported shifts in teacher understandings of the role of computational thinking in content area classrooms, as well as their self-efficacy and beliefs regarding CT integration into disciplinary content. We conclude by offering implications for the use of this model to increase teacher and student access to computational thinking practices in middle and high school classrooms.

Pair Programming is often employed in educational settings as a means of promoting collaboration and scaffolding the assignment difficulty for teams. While much research supports its inclusion as a pedagogical practice at the university level, some research has demonstrated in K-12 contexts, it can potentially lead to inequitable learning enviroments and create dynamics between partners that might negatively effect novice learners. New block-based programming environments like Netsblox have attempted to address this by creating ways for both partners to program simultaneously, but this feature has yet to be examined in detail. In this paper, we introduce several modes of Collaboration afforded by Netsblox. This includes Pair-Separate, Pair-Together, and Partner Puzzles - a mode that Splits the necessary blocks to build the assignment between team members. From an initial pilot study involving 25 pairs of middle and high school students, we find that most pairs preferred working on assignments in the Partner Puzzle mode as it presented a fun challenge to teams. We end on recommendations for building assignments using this methodology and future research directions investigating the role of collaboration in programming

Undergraduate students often struggle to learn optimal logic proof solving strategies in Discrete Math courses, primarily because of the open-ended nature of the domain. Students can, therefore, benefit from personalized tutoring, where they can receive user-adaptive support. Over the past decade, the advancements in the field of intelligent tutoring systems (ITSs) have made it possible to provide personalized tutoring with minimal involvement of a teacher or a human expert. While such tutoring systems have the potential to augment student learning on a large scale, few intelligent tutors are made open source. Deep Thought is a logic tutor where students practice constructing deductive logic proofs. Extensive research has been conducted for 11 years to provide data-driven intelligent tutoring support in Deep Thought. The logic tutor provides adaptive support using data-driven approaches on two levels: problem level, where the tutor decides whether the student should view the next problem as a worked example or they should solve it, and step level, where the tutor decides when an unsolicited partially-worked step should be provided to the student to direct them towards optimal problem-solving strategies. We have found encouraging evidence to support that the intelligent policies in Deep Thought help undergraduate students learn logic. Deep Thought is currently being used in discrete math classes at two universities: North Carolina State University, and the University of North Carolina at Charlotte. Our aim is to make this tutor available to a larger audience so as to contribute to the Computer Science Education community.

Motivated by the recent advances of reinforcement learning and the traditional grounded Self Determination Theory (SDT), we explored the impact of hierarchical reinforcement learning (HRL) induced pedagogical policies and data-driven explanations of the HRL-induced policies on student experience in an Intelligent Tutoring System (ITS). We explored their impacts first independently and then jointly. Overall our results showed that 1) the HRL induced policies could significantly improve students' learning performance, and 2) explaining the tutor's decisions to students through data-driven explanations could improve the student-system interaction in terms of students' engagement and autonomy.

This paper illustrates coding decisions by in-service teachers and high school interns working independently versus collaboratively to build computing activities for non-computing classrooms. We investigate code written in Snap! to gain insights on project type and subject matter. We also share case studies on how intern collaboration influences final product execution. Through our research, we found student-only teams often created tutorial projects whereas teachers-only teams create interactive narratives. We found students were able to reuse code across projects to replicate similar mechanics and that students specialize in different aspects of project creation. Overall, we find it beneficial to have collaborative teacher-student teams.

Many K-12 CS education instructors are new to teaching computing and use block-based programming (BBP), allowing them and their students to learn computing concepts with less frustration. Based on prior work with instructors, we found that they need assistance with grading student projects as they are new to coding and developing corresponding assessments. Existing methods are either not intuitive and too restrictive for custom assignments, or tediously unnecessary. Additionally, providing a grading support tool will help increase adoption of our Advanced Placement Computer Science Principles curriculum. Therefore, we began developing a tool to assist teachers with grading BBP projects. This paper reports on a modified User Experience Research method, which we used to determine the critical and necessary features needed for the tool to allow teachers to successfully and efficiently assess BBP student artifacts.

We present a new framework describing how teachers use ST Math, a curriculum-integrated, year-long educational game, in 3rd-4th grade classrooms. We combined authentic classroom observations with teacher interviews to identify teacher needs and practices. Our findings extended and contrasted with prior work on teachers' behaviors around classroom games, identifying differences likely arising from a digital platform and year-long curricular integration. We suggest practical ways that curriculum-integrated games can be designed to help teachers support effective classroom culture and practice.

Parsons problems are program puzzles, where students piece together code fragments to construct a program. Similar to block-based programming environments, Parsons problems eliminate the need to learn syntax. Parsons problems have been shown to improve learning efficiency when compared to writing code or fixing incorrect code in lab studies, or as part of a larger curriculum. In this study, we directly compared Parsons problems with block-based programming assignments in classroom settings. We hypothesized that Parsons problems would improve students' programming efficiency on the lab assignments where they were used, without impacting performance on the subsequent, related homework or the later programming project. Our results confirmed our hypothesis, showing that on average Parsons problems took students about half as much time to complete compared to equivalent programming problems. At the same time, we found no evidence to suggest that students performed worse on subsequent assignments, as measured by performance and time on task. The results indicate that the effectiveness of Parsons problems is not simply based on helping students avoid syntax errors. We believe this is because Parsons problems dramatically reduce the programming solution space, letting students focus on solving the problem rather than having to solve the combined problem of devising a solution, searching for needed components, and composing them together.

Learning optimal policies in real-world domains with delayed rewards is a major challenge in Reinforcement Learning. We address the credit assignment problem by proposing a Gaussian Process (GP)-based immediate reward approximation algorithm and evaluate its effectiveness in 4 contexts where rewards can be delayed for long trajectories. In one GridWorld game and 8 Atari games, where immediate rewards are available, our results showed that on 7 out 9 games, the proposed GP-inferred reward policy performed at least as well as the immediate reward policy and significantly outperformed the corresponding delayed reward policy. In e-learning and healthcare applications, we combined GP-inferred immediate rewards with offline Deep Q-Network (DQN) policy induction and showed that the GP-inferred reward policies outperformed the policies induced using delayed rewards in both real-world contexts.

In summer 2018, we conducted two week-long professional development workshops for 116 middle and high school teachers interested in infusing computational thinking (CT) into their classrooms. Teachers learned to program in Snap!, connect CT to their disciplines, and create infused CT learning segments for their classes. This paper investigates the extent to which teachers were able to successfully infuse CT skills of pattern recognition, abstraction, decomposition, and algorithms into their learning products.

Computational Thinking (CT) is being infused into curricula in a variety of core K-12 STEM courses. As these topics are being introduced to students without prior programming experience and are potentially taught by instructors unfamiliar with programming and CT, appropriate lesson design might help support both students and teachers. "Use-Modify-Create" (UMC), a CT lesson progression, has students ease into CT topics by first "Using" a given artifact, "Modifying" an existing one, and then eventually "Creating" new ones. While studies have presented lessons adopting and adapting this progression and advocating for its use, few have focused on evaluating UMC's pedagogical effectiveness and claims. We present a comparison study between two CT lesson progressions for middle school science classes. Students participated in a 4-day activity focused on developing an agent-based simulation in a block-based programming environment. While some classrooms had students develop code on days 2-4, others used a scaffolded lesson plan modeled after the UMC framework. Through analyzing student's exit tickets, classroom observations, and teacher interviews, we illustrate differences in perception of assignment difficulty from both the students and teachers, as well as student perception of artifact "ownership" between conditions.

The Advanced Placement Computer Science Principles (AP CSP) course has been fully active for 2 years, garnering a large group of diverse students [2], and flaming the need for highly trained CSP teachers, especially in effective practices for diversity and equity. We have conducted summer professional development (PD) workshops from 2012-2018 which have prepared 748 teachers to teach AP CSP using the Beauty and Joy of Computing (BJC) curriculum. To improve equity and readiness for teaching, we have refined our PD by: shortening the PD from 6 weeks to 1 week; developing new, highly scaffolded pre-PD work; and modifying the in-person schedule to incorporate more pedagogy and teaching experiences, while continuing to provide in-depth, hands-on support for teachers to learn the basics of programming. The most recent revisions to our PD schedule resulted in improved post-PD survey results, with teachers from the 2018 cohort planning to adopt more of the BJC curriculum than in past years. From 2017 to 2018, planned adoption rates increased by 13%, resulting in 73% of the 2018 PD participants planning to adopt more than $60%$ of the BJC curriculum and 58% planning to adopt 80-$100% of the BJC curriculum in their classrooms in 2018-2019. In this paper, we discuss the most recent BJC PD implementation and provide evidence of increased teacher self-efficacy in areas including fostering interest in computing for underrepresented populations.

Automated grading has become crucial in supporting large introductory Computer Science courses by assisting instructors in reducing grading time and course costs. However, novice programmers are often frustrated by auto-grading tools as they often provide minimal feedback or the questions are too complex. We propose using Bloom's Taxonomy to gradually increase the complexity of the programming question in an auto-grading tool. The easiest questions are based on Knowledge and Comprehension. Next are questions that require Application and Analysis. The most complex questions require Design and Creativity. We have developed programming questions that fit these categories. For example, for debugging the students have to understand and Analyze code. However, in order to write a program with more complex logic the students have to be Creative. The main goal is to lead novice programmers to learn more effectively and efficiently. In this poster, we present examples of the developed auto-graded programming problems based on Bloom's Taxonomy and the results of a pilot study in a CS-1 non-majors course

One way to increase access to education on computing is to integrate computational thinking (CT) into K12 disciplinary courses. However, this challenges teachers to both learn CT and decide how to best integrate CT into their classes. In this position paper, we present PRADA, an acronym for Pattern Recognition, Abstraction, Decomposition, and Algorithms, as a practical and understandable way of introducing the core ideas of CT to non-computing teachers. We piloted the PRADA model in two, separate, week-long professional development workshops designed for in-service middle and high school teachers and found that the PRADA model supported teachers in making connections between CT and their current course material. Initial findings, which emerged from the analysis of teacher-created learning materials, survey responses, and focus group interviews, indicate that the PRADA model supported core content teachers in successfully infusing CT into their existing curricula and increased their self-efficacy in CT integration.

Tinkering has been shown to have a positive influence on students in open-ended making activities. Open-ended programming assignments in block-based programming resemble making activities in that both of them encourage students to tinker with tools to create their own solutions to achieve a goal. However, previous studies of tinkering in programming discussed tinkering as a broad, ambiguous term, and investigated only self-reported data. To our knowledge, no research has studied student tinkering behaviors while solving problems in block-based programming environments. In this position paper, we propose a definition for tinkering in block-based programming environments as a kind of behavior that students exhibit when testing, exploring, and struggling during problem-solving. We introduce three general categories of tinkering behaviors (test-based, prototype-based, and construction-based tinkering) derived from student data, and use case studies to demonstrate how students exhibited these behaviors in problem-solving. We created the definitions using a mixed-methods research design combining a literature review with data-driven insights from submissions of two open-ended programming assignments in iSnap, a block-based programming environment. We discuss the implication of each type of tinkering behavior for learning. Our study and results are the first in this domain to define tinkering based on student behaviors in a block-based programming environment.

Program Visualizations (PVs) have been used as educational tools to allow students to visually inspect the runtime behavior of their code. However, many of these systems act as low-level visual debuggers not high-level abstractions of program behavior. Additionally, evaluations of these systems tend to focus more on student engagement or opinion in using the system and not on artifacts produced using the system. This paper discusses the effectiveness of a PV developed to aide students in an undergraduate Artificial Intelligence class on a pathfinding homework assignment. Students in 4 semesters of the course were tasked to develop pathfinding algorithms for an agent to navigate worlds in cases of both certain and uncertain world information. Students in 2 semesters of the course were given access to a PV that allowed them to see a visual representation of their agent navigating the world in either information condition. The final agents developed by these students were compared with those developed by students who never received the PV. Comparisons were made on the performance of these agents in both cases of uncertain and certain world information on several test worlds. Student written reports for the Experimental condition were also analyzed. The results showed significant differences in the performance of the algorithms developed in both certain and uncertain world information. Student reflections on using the PV within the written reports provide insight into how the PV informed the design and development of their submission.

Research in a variety of domains has shown that viewing worked examples (WEs) can be a more efficient way to learn than solving equivalent problems. We designed a Peer Code Helper system to display WEs, along with scaffolded self-explanation prompts, in a block-based, novice programming environment called \snap. We evaluated our system during a high school summer camp with 22 students. Participants completed three programming problems with access to WEs on either the first or second problem. We found that WEs did not significantly impact students' learning, but may have impacted students' intrinsic cognitive load, suggesting that our WEs with scaffolded prompts may be an inherently different learning task. Our results show that WEs saved students time on initial tasks compared to writing code, but some of the time saved was lost in subsequent programming tasks. Overall, students with WEs completed more tasks within a fixed time period, but not significantly more. WEs may improve students' learning efficiency when programming, but these effects are nuanced and merit further study.

With increased numbers of K-12 computing courses, we also see an increase in teachers new to the subject, making it difficult for them to properly assess student programming assignments. Many of these teachers require project-specific rubrics to help assess student learning. Researchers have attempted to create systematic, validated, and reliable rubrics for these courses with only minor success. In this research, we make an argument for the validity of our low-stakes computing rubrics. In doing so, we establish a validated method for creating a full-suite of project-based rubrics for K-12 computing courses, helping teachers, researchers, and practitioners make much-needed course materials. Evaluating these rubrics, we see grader consistency as well as heatmaps of where teachers are looking for computational thinking concepts in code.

Although there has been much work on procedural content generation for other game genres, very few researchers have tackled automated content generation for educational games. In this paper, we present a template-based, automatic puzzle generator for an educational puzzle programming game called BOTS. Two experts created their own new puzzles and evaluated generator-generated puzzles for meeting the educational goals, the structural and visual novelty. We show that our generator can generate puzzles with expert-designed educational goals while saving experts more than 80% of creation time, and these puzzles exhibit structural and visual novelty compared to expert-created puzzles. The contribution of this work is defined and implemented the first template-based automatic puzzle generator that saves expert time while incorporating expert-designed educational goals and enhancing puzzle creativity.

When novice students encounter difficulty when learning to program, some can seek help from instructors or teaching assistants. This one-on-one tutoring is highly effective at fostering learning, but busy instructors and large class sizes can make expert help a scarce resource. Increasingly, programming environments attempt to imitate this human support by providing students with hints and feedback. In order to design effective, computer-based help, it is important to understand how and why students seek and avoid help when programming, and how this process differs when the help is provided by a human or a computer. We explore these questions through a qualitative analysis of 15 students' interviews, in which they reflect on solving two programming problems with human and computer help. We discuss implications for help design and present hypotheses on students' help-seeking behavior.

Growing public demand for computer science (CS) education in K-12 schools requires an increase in well-qualified and well-supported computing teachers. To alleviate the lack of K-12 computing teachers, CS education researchers have focused on hosting professional development workshops to prepare in-service teachers from other disciplines to teach introductory level computing courses. In addition to the curriculum knowledge and pedagogical content knowledge taught in the professional development workshops, these new teachers need support in computer science subject matter knowledge throughout the school year. In particular, these new teachers find it difficult to grade programs and labs. This research study uses two variations of the Delphi Method to create learning-oriented rubrics for Computer Science Principles teachers using the Beauty and Joy of Computing curriculum. To perform this study we implemented (1) a heavy-weight, heterogeneous wide-net Delphi, and (2) a lower-weight, homogeneous Delphi composed of master teachers. These methods resulted in the creation of two systematically- and rigorously-created rubrics that produce consistent grading and very similar inter-rater reliabilities.

This showpiece will present iSnap, an extension of the block-based, novice programming environment Snap!, which supports struggling students by providing on-demand hints and feedback that help them complete programming assignments. iSnap extends the existing syntactic scaffolding offered by block-based programming to additionally support the implementation of programming tasks. Research on iSnap has explored questions of how visual programming environments can better support learners, the impact of this support, and how learners seek and use computer-based help. The showpiece will consist of an interactive demonstration of iSnap, including the user interface experienced by students and the data-driven algorithm used to automatically generate the programming feedback.

Frame-based editing is a novel way to edit programs, which claims to combine the benefits of textual and block-based programming. It combines structured `frames' of preformatted code, designed to reduce the burden of syntax, with `slots' that allow for efficient textual entry of expressions. We present an empirical evaluation of Stride, a frame-based language used in the Greenfoot IDE. We compare two groups of middle school students who worked on a short programming activity in Greenfoot, one using the original Java editor, and one using the Stride editor. We found that the two groups reported similarly low levels of frustration and high levels of satisfaction, but students using Stride progressed through the activity more quickly and completed more objectives. The Stride group also spent significantly less time making purely syntactic edits to their code and significantly less time with non-compilable code.

The "Beauty and Joy of Computing" Computer Science Principles class has inspired many new teachers to learn to teach creative computing classes in high schools. However, new computer science teachers feel under-prepared to grade open-ended programming assignments and support their students' successful learning. Rubrics have widely been used to help teaching assistants grade programs and are a promising way to support new teachers to learn how to grade BJC programs. In this paper, we adapt general coding criteria from auto-graders to a lab where students write code to draw a brick wall. We tested the rubric on student assignments and showed that we can achieve high inter-rater agreement with the refined rubric.

Our previous work has demonstrated that players who perceive a game as more challenging are likely to perceive greater learning from that game [8]. However, this may not be the case for all sources of challenge. In this study of a Science learning game called Quantum Spectre, we found that students' progress through the first zone of the game seemed to encounter a "roadblock" during gameplay, dropping out when they cannot (or do not want to) progress further. Previously we had identified two primary types of errors in the learning game, Quantum Spectre: Science Errors related to the game's core educational content; and Puzzle Errors related to rules of the game but not to science knowledge. Using this prior analysis, alongside Survival Analysis techniques for analyzing time-series data and drop-out rates, we explored players' gameplay patterns to help us understand player dropout in Quantum Spectre. These results demonstrate that modeling player behavior can be useful for both assessing learning and for designing complex problem solving content for learning environments.

Data-driven methods have previously been used in intelligent tutoring systems to improve student learning outcomes and predict student learning methods. We have been incorporating data-driven methods for feedback and problem selection into Deep Thought, a logic tutor where students practice constructing deductive logic proofs. In this latest study we have implemented our data-driven proficiency profiler (DDPP) into Deep Thought as a proof of concept. The DDPP determines student proficiency without expert involvement by comparing relevant student rule scores to previous students who behaved similarly in the tutor and successfully completed it. The results show that the DDPP did improve in performance with additional data and proved to be an effective proof of concept.

Increasing need for computing expertise in our everyday lives and in the workforce, paired with declining enrollments in computing by women and underrepresented minorities have made it critical to provide students with experiences in computing before college. CS10K is a national effort to engage 10,000 high school teachers in teaching computer science across the United States. With CS10K projects working on this goal since 2012 having prepared less than 1000 teachers for teaching computer science in high school, there is a need to scale professional development opportunities to local communities. College and university computing faculty have the unique preparation and call for community engagement that make it a win-win to support local high school teachers in learning to teach computer science. We will use the STARS model of building university-based communities that broaden participation in computing, adapted to the context of supporting K12 teachers to become change agents and educators prepared to teach computer science. In this workshop, participants will learn how to implement scalable team-based professional development for K12 teachers new to teaching computer science. We will provide resources to recruit, plan, and support small groups of new teachers to teach the new CS Principles course that will become an Advanced Placement course in 2016-2017. Laptops for attendees are optional.

The Beauty and Joy of Computing (BJC) is a CS Principles (CSP) course developed at UC Berkeley, intended for high school juniors through university non-majors. It was twice chosen as a CSP pilot, and both the College Board and code.org have endorsed it. Since 2011, we have offered professional development to over 240 high school teachers. Our guiding philosophy is to meet students where they are, but not to leave them there. It covers the big ideas and computational thinking practices required in the AP CSP curriculum framework using an easy-to-learn blocks-based programming language called Snap! (based on Scratch), and powerful computer science ideas like recursion, higher-order functions and computability. Through the course, students learn to create beautiful images, and realize that code itself can be beautiful. Having fun is an explicit course goal. We take a "lab-centric" approach, and much of the learning occurs through guided programming labs that ask students to explore and play. In this workshop, we will provide an overview of BJC, share our experiences as instructors of the course at the university and high school level, provide a glimpse into a typical week of the course, and share details of potential crowd-funded summer professional development opportunities. This is a hands-on workshop. Laptops are required, and all "handouts" will be digital.

The STARS Computing Corps is a national alliance with the mission to grow a diverse community of computing leaders. STARS serves as a framework for integrating civic engagement into college computing departments with the goals of broadening participation of underrepresented groups in computing, recruiting K-12 students into the computing pipeline, and retaining students in computing majors. The STARS approach to broadening the participation of women and underrepresented minorities in computing is based on research that has shown the value of creating a community and sense of identity. The Corps creates such a community across multiple institutions, including women's and historically black colleges and universities, with members that share the core values of becoming responsible leaders who use their computing skills for social benefit. Each cohort of students and faculty at a STARS member university collaborates with local K-12 schools and industry partners to conduct computing-related outreach, service, and research that can broaden participation in computing. These local cohorts help to build community within and across STARS member institutions, retain students in college degree programs in computing, recruit new students into computing, educate local K-12 teachers, counselors, students, and parents about computing, and build bridges with local industry and community organizations. This workshop will provide hands-on training for new schools to learn how to begin and build a STARS Computing Corps cohort on their own campus.

Computer Science Principles (CSP) will become an Advanced Placement course during the 2016-17 school year, and there is an immediate need to train new teachers to be leaders in computing classrooms. From 2012-2015, the Beauty and Joy of Computing team offered professional development (PD) to 133 teachers, resulting in 89 BJC CSP courses taught in high schools. Our data show that the PD improved teachers' confidence in our four core content categories and met its primary goal of training teachers in equitable, inquiry-based instruction. In this paper, we present the evolution of the BJC PD, its challenges and lessons that we learned while continually adapting to teachers' needs and contexts.

Visual, block-based programming environments present an alternative way of teaching programming to novices and have proven successful in classrooms and informal learning settings. However, few studies have been able to attribute this success to specific features of the environment. In this study, we isolate the most fundamental feature of these environments, the block interface, and compare it directly to its textual counterpart. We present analysis from a study of two groups of novice programmers, one assigned to each interface, as they completed a simple programming activity. We found that while the interface did not seem to affect users' attitudes or perceived difficulty, students using the block interface spent less time off task and completed more of the activity's goals in less time.

Intelligent tutoring systems and other computer-aided learning environments produce large amounts of transactional data on student problem-solving behavior, in previous work we modeled the student-tutor interaction data as a complex network, and successfully generated automated next-step hints as well as visualizations for educators. In this work we discuss the types of tutoring environments that are best modeled by interaction networks, and how the empirical observations of problem-solving result in common network features. We find that interaction networks exhibit the properties of scale-free networks such as vertex degree distributions that follow power law. We compare data from two versions of a propositional logic tutor, as well as two different representations of data from an educational game on programming. We find that statistics such as degree assortativity and the scale-free metric allow comparison of the network structures across domains, and provide insight into student problem solving behavior.

We have developed a novel data-driven mastery learning system to improve learning in complex procedural problem solving domains. This new system was integrated into an existing logic proof tool, and assigned as homework in a deductive logic course. Student performance and dropout were compared across three systems: The Deep Thought logic tutor, Deep Thought with integrated hints, and Deep Thought with our data-driven mastery learning system. Results show that the data-driven mastery learning system increases mastery of target tutor-actions, improves tutor scores, and lowers the rate of tutor dropout over Deep Thought, with or without provided hints.

In this poster, we describe efforts to assess computational thinking activities that can be easily implemented in any science classroom. Studies have shown that a set of conditions must be met for computational thinking tools to be used in K-12 education and that when they are used, there is a wide spectrum in the level of computational thinking that the tool enables. This study extends this work by examining how middle school students translated their science fair projects into Scratch and what evidence of computational thinking is present. Scrape, a tool designed to analyze Scratch projects was used. Overall, it was found that most students simply created a presentation of their project without much complexity. Eight students created interactive projects that required user participation and used more advanced computational concepts. Finally, recommendations are given for next steps in the creation of a series of activities that would scaffold student learning as they apply to computational thinking concepts of a science concept.

Students often take computing classes because they are eager to create games, to learn to create meaningful and useful software, or both. Connecting computing to real, cutting-edge applications has been shown to increase engagement of women and minorities. The new CS Principles curriculum, a pilot Advanced Placement course, seeks to broaden the participation in computing to a larger and more diverse audience. This curriculum emphasizes that computing is a creative activity where people work together to solve relevant problems. In this workshop, we introduce free software and curricula to enable novice high school and college students in a first computing course to learn basic game and mobile phone development. We discuss how these activities facilitate teaching high school and non-major (CS0) course topics, but they can also be used to illustrate more advanced topics. Participants will learn GameMaker and mobile phone programming using AppInventor, and/or Touch Develop. These tools allow students to create and have fun with computing while teaching object-oriented and event-driven programming and game architectures. We will provide links to curricular modules for the CS Principles: Beauty and Joy of Computing course, as well as links to the GameMaker, AppInventor, and Touch Develop platforms and tutorials. Participants must bring a network-connected laptop with a modern browser, and the latest version of Java, and may optionally bring an Android, Windows, or iPhone.

The new CS Principles curriculum, a pilot Advanced Placement course, offers novice students an exciting opportunity to learn computing in a hands-on, fun way. High school and college teachers of introductory computer science course are invited to this workshop to learn basic game and mobile phone development. Participants will learn GameMaker, AppInventor, and Touch Develop. These tools allow students to create and have fun with computing while teaching object-oriented and event-driven programming and game architectures. Participants should bring their own laptops (ideally with AppInventor installed). Windows 7 phones will be provided during the workshop. We will provide links to curricular modules for the CS Principles: Beauty and Joy of Computing course.

This design research paper examines the implementation and curriculum changes of an after school computer science program that promotes computational thinking to middle school students. The program, Students in Programming, Robotics, and Computer Science (SPARCS), can adapt to different presentation environments, such as independent after school sessions or a semester-long apprenticeship program. We trace one implementation of the program through the initial deployment, the development of infrastructure, and a reorganization of content to address student interests. We found that student attrition dropped and the average session enjoyment increased when our sessions integrated consumer technologies such as mobile applications, video games, and the Minecraft computer game. In this paper, we provide readers a framework for running computing outreach activities around similar consumer technologies.

The Beauty and Joy of Computing (BJC) is an introductory computer science curriculum developed at UC Berkeley (and adapted at the University of North Carolina, Charlotte), intended for high school juniors through university non-majors. It was used in two of the five initial pilot programs for the AP CS Principles course being developed by the College Board and the National Science Foundation. Our overall goal is to support the CS10K project by preparing instructors to teach the AP CS Principles course through the BJC curriculum. In this workshop, we will share our experiences as instructors of the course at the university and high school level, provide a glimpse into a typical week of the course, and share details of NSF-funded summer professional development opportunities.

This workshop is for high school and college teachers of general-interest ("CS 0") computer science courses, especially the AP CS: Principles course. SNAP! (Build Your Own Blocks) is a free, browser-based, graphical, drag-and-drop language inspired by Scratch. The beauty of the Scratch programming environment, designed for 8-14 year olds, is that it makes abstract concepts more concrete and understandable to a broader audience. SNAP! extends Scratch to support older learners (14-20) with built-in named procedures (thus recursion), procedures as data (thus higher order functions), structured lists, and sprites as first class objects with inheritance.

The demand for computing professionals in the U.S. workforce is expected to increase over the next several years, while the number of students intending to major in computing has declined. In this BOF session, we focus on the use of service learning to address the growing concern of creating a sustainable pipeline for computing professionals, with an emphasis on broadening participation in computing. We center the discussion around our experiences with the STARS Computing Corps, which engages college students in service-learning to address the national computing talent shortage. The model has been shown to be effective in retaining college students in computing, and STARS alumni credit participation in STARS with helping to develop professional skills. This session will include interactive small group discussions on goals, obstacles, and strategies for applying service learning to broaden participation in computing, grow the K-12 pipeline, and prepare college students with skills demanded in the 21st century workforce.

Students often take computing classes because they are eager to create games, to learn to create meaningful and useful software, or both. Connecting computing to real, cutting-edge applications has been shown to increase engagement of women and minorities. The new CS Principles curriculum, a pilot Advanced Placement course, seeks to broaden the participation in computing to a larger and more diverse audience. This curriculum emphasizes that computing is a creative activity where people work together to solve relevant problems. In this workshop, we introduce free software and curricula to enable novice high school and college students in a first computing course to learn basic game and mobile phone development. We discuss how these activities facilitate teaching high school and non-major (CS0) course topics, but they can also be used to illustrate more advanced topics. Participants will learn GameMaker and mobile phone programming using AppInventor, and/or Touch Develop. These tools allow students to create and have fun with computing while teaching object-oriented and event-driven programming and game architectures. If possible, Windows 7 phones will be provided for use during the workshop. We will provide links to curricular modules for the CS Principles: Beauty and Joy of Computing course, as well as links to the GameMaker, AppInventor, and Touch Develop platforms and tutorials. Participants must bring a network-connected laptop with a modern browser, and the latest version of Java (ideally with AppInventor installed), and may optionally bring an Android or Windows 7 phone.

The Beauty and Joy of Computing (BJC) is an introductory computer science curriculum developed at UC Berkeley (and adapted at the University of North Carolina, Charlotte), intended for high school juniors through university non-majors. It was used in two of the five initial pilot programs for the AP CS Principles course being developed by the College Board and the National Science Foundation. Our overall goal is to support the CS10K project by preparing instructors to teach the AP CS Principles course through the BJC curriculum. In this workshop, we will share our experiences as instructors of the course at the university and high school level, provide a glimpse into a typical week of the course, and share details of NSF-funded summer professional development opportunities.

This workshop is for high school and college teachers of general-interest ("CS 0") CS courses. It presents the programming environment used in two of the five initial AP CS Principles pilot courses.

Examining student interactions in multi-step problems from Intelligent Tutoring Systems currently involves examining thousands of interactions from hundreds of students. We designed and implemented a sequential pattern mining algorithm and a sequence rating algorithm that together recognize interesting student action sequences and display them to the user in the context of the larger graph system. With the added feature of our algorithms in the InVis system, we hope to allow teachers and tutoring system designers to better understand student action patterns and thus cater better to their learning.

Social capital implies that social networks have value. It is therefore important that when a person is at an academic conference, they must strive to build a strong professional social network for themselves. This can be difficult for many academic conference attendeees. We present Table Tilt, a two-minute ice-breaker game for 2--6 players with iPhones or iPods, that was built to facilitate team building and help individuals build social capital. Table Tilt leverages human sociality and game rules to promote communication and teamwork.

Providing users the ability to create their own unique content in educational software and games can be highly effective at motivating the users to use and reuse the system. It is especially popular with students who self identify as creative or wanting to do their own thing rather than a prescribed activity. Due to the popularity of user generated content modes, some users may ignore other modes the software has to offer and only create new original content. Therefore it is important to maximize the learning potential and effectively guide user behavior in a constructivist free play environment. However, in doing so it is vital that we do not hamper the creative freedom of the user, the very reason users enjoy content creation. Here we present effective strategies for meeting these goals, provide an example implementation, and present results of a study using the example.

We present a curriculum for computer science outreach using Game Maker. This curriculum has been adapted over six iterations of a 10-week, middle school apprenticeship on Game Design and Development. Through multiple iterations we have adjusted for many of the issues one could expect to encounter when running a similar outreach program. While many outreach curricula are independent from coursework, our video game design curriculum is targeted to address generalized student learning objectives (Math and English Language Arts) and designed for integration into middle school classrooms. We demonstrate that students' language arts and math classroom performance have improved with participation in this apprenticeship.

We present the Game2Learn methodology for the design and evaluation of educational games with a focus on well-defined learning objectives and empirical verification. This integrative process adapts ideas from educational design, intelligent tutoring systems, classical test-theory, and interaction and game design, and agile software development. The methodology guides researchers through the steps of the design process, including identification of specific learning objectives, translation of learning activities to game mechanics, and the empirical evaluation of the final product. This methodology is particularly useful for ensuring successful student research experiences or software engineering courses.

The Beauty and Joy of Computing (BJC) is an introductory computer science curriculum developed at the University of California, Berkeley (and adapted at the University of North Carolina, Charlotte), intended for high school juniors through university non-majors. It was used in two of the five initial pilot programs for the AP CS Principles course being developed by the College Board and the National Science Foundation. Our overall goal is to support the CS10K project by preparing instructors to teach the AP CS Principles course through the BJC curriculum. In this workshop, we will share our experiences as instructors of the course at the university and high school level, provide a glimpse into a typical week of the course, and share details of NSF-funded summer professional development opportunities.

The College Board's guidelines for the coming AP CS Principles course are broad enough to allow many different interpretations. In particular, different courses have different levels of technical depth. The "Beauty and Joy of Computing" curriculum, used by two of the initial five pilot sites, aims high, with recursion and higher order functions included in the programming half of the course. This session is for high school or college level instructors considering teaching an AP CS Principles course and interested in using the BJC curriculum, and/or the Snap! (formerly BYOB) visual programming language used in the curriculum. See http://bjc.berkeley.edu for the curriculum and http://snap.berkeley.edu for the language.

The prevalence of obesity among adolescents and adults in the U.S. is a matter of concern. Exercise video games reach a wide audience and can be used to motivate increased physical activity. We have previously developed Astrojumper, an exergame exploring game mechanics that provide a fun experience and effective exercise, and have now developed a new version of Astrojumper that supports interval training through additional mechanics. We believe the new version will improve upon the first in player motivation, enjoyment and replayability, and also in the level of physical challenge the game affords its players.

BeadLoom Game (BLG) is an educational puzzle game developed by adding game elements to a free-play educational tool called the Virtual Bead Loom (VBL). To motivate students who prefer the creative freedom of VBL, we added Custom Puzzle mode to BLG so players can create, share, and rate user-generated puzzles. We compare VBL and BLG Custom Puzzles to show that this mode increases the creativity and complexity of student work.

BeadLoom Game (BLG) is an educational puzzle game designed to teach students basic Cartesian coordinates, iteration, and layering. Although this game has been proven to be successful at teaching students these concepts, many participants reported wanting more competitive and free-play creative elements in the game. In response, we augmented the BeadLoom Game with a competitive high score table, a creative custom puzzle mode, and a social network framework. Here we report results of an experiment where middle school students are given versions of the BLG with different combinations of these new features. Based on the in-game metrics and player surveys we show that while both the competitive and the creative game modes increase a majority of the player's motivation it is not until we add both features that we maximize this effect. Through a combination of creative and competitive game modes we are able to have the highest motivation for the largest number of different players.

The Virtual Bead Loom (VBL) is a Culturally Situated Design Tool that successfully teaches students middle school math concepts while they learn about and create their own Native American bead artifacts. We developed BeadLoom Game to augment VBL with game elements that encourage players to apply the computational thinking skills of iteration and layering while optimizing the number of steps they take to solve a puzzle. In our prior work, we showed that BeadLoom Game is effective at teaching Cartesian coordinates, iteration, and layering. In this study, we use a switching replications experimental design to compare performance of BeadLoom Game with the VBL. Our results from two summer camps, one for middle school and one for college-bound high school students, show that through the addition of game based objectives, BeadLoom Game teaches Cartesian coordinates as well as the VBL but also teaches the computational thinking practices of iteration and layering.

Since 2008, NSF and The College Board, have been developing a "Computer Science: Principles" curriculum to "introduce students to the central ideas of computing and CS, to instill ideas and practices of computational thinking, and to have students engage in activities that show how computing and CS change the world". We report on the initial pilot of the CS Principles curriculum at 5 universities in 2010/11. The instructors from the pilot schools will describe their classes, the piloting experience (teaching under a microscope), and successes and failures. Emphasis will be on: mapping the CS Principles curriculum to a college's specific needs, and how others can use or modify the existing materials for pilots at their schools.

The Students and Technology in Academia, Research, and Service (STARS) Alliance is a nationally-connected system of regional partnerships among higher education, K-12 schools, industry and the community with a mission to broaden the participation of women, under-represented minorities and persons with disabilities in computing (BPC). Each regional partnership is led by a STARS member college or university with partners such as local chapters of the Girl Scouts, the Black Data Processors Association, public libraries, Citizen Schools, and companies that employ computing graduates. STARS goals include retaining and graduating undergraduates and recruiting and bridging undergraduates into graduate programs. The alliance works toward these goals through activities that advance the central values of Technical Excellence, Leadership, Community, and Service and Civic Engagement. In particular, all STARS college and university members implement the STARS Leadership Corps (SLC), an innovative model for enveloping a diverse set of BPC practices within a common framework for implementation within multiple organizations, common assessment, and sustainability through curricula integration. Herein, we describe the SLC model and its implementation in the STARS schools, including details of an SLC service-learning course that has been adopted by eight STARS schools. We report the results of our three-year study of the SLC in the 20 STARS schools. Our study found a positive effect of participation in the SLC on important student success variables, including self-efficacy, perceived social relevance of computing, grade point average, and commitment to remain in computing. Results indicate that the SLC model is effective for students under-represented in computing, as well as for those not from under-represented groups.

The Virtual Bead Loom (VBL) was designed to teach mathematical concepts such as Cartesian coordinates, symmetry, and iteration to middle and high school math students through the design of Native American-inspired bead loom art. In our outreach programs using the VBL, we noted that the students avoid using complex functions such as iteration, instead creating designs one point or line at a time. To motivate students to learn the advanced concepts, we created the BeadLoom Game by adding game elements to the VBL. We have tested the BeadLoom Game with two summer camps and found that the game motivates students, exposes them to more complex computing-related math concepts, and increases the chance that students will continue using the tool beyond assigned class time.

Serious games are an exciting new research area that combines expertise across a wide range of computing skills, from programming and software engineering to algorithms, problem solving, and networking with design skills. Teaching computing students to create effective games with a serious purpose within a semester can be quite challenging, even on a one on one basis. We present the structure, format, and outcomes from an experimental course in serious games research and prototyping conducted at the University of North Carolina at Charlotte.

Today's Millennial students have changing preferences for education and work environments that negatively affect their enrollment and retention rates into university computer science programs. To better suit these preferences, and to improve CS educational techniques, teaching methods and tools outside of the traditional lecture sessions and textbooks must be explored and implemented. Currently, both serious games and collaborative classroom work, including pair programming, are the focus of studies meant to do just this. The proposed work deals with both serious games and student collaboration research, positing that educational games with collaborative elements (multiplayer games) will take advantage of the benefits offered by each of these areas, resulting in an educational game that demonstrates increased learning gains and student engagement above that of individual learning game experiences. Collaborative educational games and software also have the potential to solve many of the problems that collaborative work may pose to course instructors in terms of helping to regulate and evaluate student performance.

One of the primary goals of attending academic conferences is professional networking, yet even though this interaction can increase one's feeling of community within a field, conference attendees are not interacting as much as they could be. Similarly, it's known that students who do not feel as if they are part of a larger academic community are less likely to participate in extracurricular activities and organizations, lowering retention rates. To combat both of these problems, we present SNAG (Social Networking and Games). SNAG is a suite of mobile and Internet games which aim to facilitate social networking between members of a group, and can be used in either a conference setting or within a university. This paper focuses on one specific game, Snag'em, and discusses our evaluation for our SNAG games.

Projects funded by the National Science Foundation (NSF) Research Experiences for Undergraduates (REU) program aim to (a) enhance participation of students who otherwise might not have research opportunities, and (b) increase the number of students interested in graduate programs, thus expanding the pool of a well-trained scientific workforce. To provide meaningful experiences for these students, REU projects make use of a set of interesting, appropriate research problems that can be tackled in 8 to 10 weeks in summer.

Children with autism show substantial benefits from rigorous physical activity, however it is often difficult to motivate these individuals to exercise due to their usually sedentary lifestyles. To address the problem of motivation, we have developed Astrojumper, a stereoscopic virtual reality exergame which was designed to fit the needs of children with autism. During the game, virtual space-themed objects fly forward toward the user who must use their own physical movements to avoid collisions. Preliminary playtesting of Astrojumper on neuro-typical participants has been positive, and we plan to run an extensive evaluation assessing the psychological and physiological effects of this system on children with and without autism.

We have developed a tool for generating hints within computer-aided instructional tools based on a corpus of student work. This tool allows us to select source problem solutions that match the current user solution and generate hints based on next problem steps that are most likely to lead to a successful solution. However, within such a tool it is possible to generate hints that did not turn out to be useful in the source problem solution. Therefore, we have proposed a metric to measure and integrate a “utility” function to choose source material for hint generation. In this paper we present our metric and an experiment to investigate its use on real data from a logic proof tutorial.

We have developed a learning game to teach loops, nested loops, and arrays using scaffolding and interactive visualization. We compare the game to a traditional programming assignment in an introductory computing laboratory. In our study, 17 Introduction to Computer Science labs were randomly assigned to play the learning game first and half to write a program first. Our results show that students playing the learning game first learn more, and that students prefer the game assignment to the traditional assignment. These results suggest that incorporation of game-based assignments is beneficial to student learning.

We are developing games to increase student learning and attitudes in introductory CS courses. Wu's Castle is a game where students program changes in loops and arrays in an interactive, visual way. The game provides immediate feedback and helps students visualize code execution in a safe environment. We compared the game to a traditional programming assignment in an introductory CS course. In our study, half of the students were randomly selected to play the learning game first and half to write a program first. Our results show that students who play our learning game first outperform those who write a program before playing the game. Students in the game-first group felt they spent less time on the assignments, and all students preferred the learning game over the program. These results suggest that games like Wu's Castle can help prepare students to create deeper, more robust understanding of computing concepts while improving their perceptions of computing homework assignments.

No abstract available.

This paper presents the design of the final stage of a new game currently in development, entitled cMotion, which will use virtual humans to teach emotion recognition and programming concepts to children. Having multiple facets, cMotion is designed to teach the intended users how to recognize facial expressions and manipulate an interactive virtual character using a visual drag-and-drop programming interface. By creating a game which contextualizes emotions, we hope to foster learning of both emotions in a cultural context and computer programming concepts in children. The game will be completed in three stages which will each be tested separately: a playable introduction which focuses on social skills and emotion recognition, an interactive interface which focuses on computer programming, and a full game which combines the first two stages into one activity.

We are developing games to teach introductory computer science concepts to increase student motivation and engagement in learning to program. Wu's Castle is a two-dimensional role playing game that teaches loops and arrays in an interactive, visual way. In this game, the player interactively programs magical creatures to create armies of snowmen. The game provides immediate feedback and helps students visualize the execution of their code in a safe environment. We tested the game in a CS1 course, where students could earn extra credit to play Wu's Castle. Our results show learning gains for game players, compared both through pre- and post-tests differences and improved performance on relevant final exam questions when compared to students who did not play the game. The results of this study suggest that Wu's Castle implements good practices for teaching programming within a game.

Research experiences for undergraduates are considered an effective means for increasing student retention and encouraging undergraduate students to continue on to graduate school. However, managing a cohort of undergraduate researchers, with varying skill levels, can be daunting for faculty advisors. We have developed a program to engage students in research and outreach in visualization, virtual reality, networked robotics, and interactive games. Our program immerses students into the life of a lab, employing a situated learning approach that includes tiered mentoring and collaboration to enable students at all levels to contribute to research. Students work in research teams comprised of other undergraduates, graduate students and faculty, and participate in professional development and social gatherings within the larger cohort. Results from our first two years indicate this approach is manageable and effective for increasing students' ability and desire to conduct research.

This paper presents Game2Learn, an innovative project designed to leverage games in retaining students in computer science (CS). In our two-pronged approach, students in integrative final-year capstone courses and summer research experiences develop games to teach computer science, which, in turn, will be used to improve introductory computing courses. Our successful model for summer undergraduate research and capstone projects engages students in solving the computing retention problem, allows them to quickly create games, and instructs students in user- and learner-centered design and research methods. Results show that this method of building games to teach engages students at multiple levels, inspiring newer students that one day their homework may all be games, and encouraging advanced students to continue on into graduate studies in computing.

We investigated the effects of using immersive virtual humans to teach users social conversational verbal and non-verbal protocols in south Indian culture. The study was conducted using a between-subjects experimental design, and compared instruction and interactive feedback from immersive virtual humans against instruction based on a written study guide with illustrations of the social protocols. Participants were then tested on how well they learned the social conversational protocols by exercising the social conventions in front of videos of real people. The results of our study suggest that participants who trained with the virtual humans performed significantly better than the participants who studied from literature.

No abstract available.

Scenarios have proven effective for eliciting, describing and validating software requirements; however, scenario management continues to be a significant challenge to practitioners. One reason for this difficulty is that the number of possible relations among scenarios grows exponentially with the number of scenarios. If these relations are formalized, they can be more easily identified and supported. To provide this support, we extend the benefits of project-wide glossaries with two complementary approaches. The first approach employs shared scenario elements to identify and maintain common episodes among scenarios. The resulting episodes impose consistency across related scenarios and provide a way to visualize their interdependencies. The second approach quantifies similarity between scenarios. The resulting similarity measures serve as heuristics for finding duplicate scenarios, scenarios needing further elaboration, and scenarios which have not yet been identified yielding valuable information about how well the scenarios provide coverage of the requirements. These two approaches, integrated with a scenario database, project glossaries, configuration management, and coverage analysis, form the basis of a useful and effective strategy for scenario management and evolution.