In order to fully prepare high-quality teachers to deliver on the promise of CS for All, teacher preparation programs must ensure teacher candidates have mastered both content and the pedagogy to teach that content well. University CS departments are well-positioned to teach the content, but "methods courses" are an important source of pedagogical and pedagogical content knowledge for future computer science teachers. This panel will describe a variety of approaches: Colleges of Education or CS departments; in-person, hybrid, or online course delivery; and standalone "CS Methods" courses or CS integrated in broader methods courses. Each panelist will describe their course. The moderator will elicit common features between the courses as well as the strengths and weaknesses between different approaches.

Over a decade ago, the CS1 course for students without prior programming experience at a large research-intensive university was redesigned to incorporate three best practices in teaching programming: Media Computation, Pair Programming, and Peer Instruction. The purpose of this revision was to improve the quality of the course, appeal to a larger student body, and improve retention in the major. An initial analysis of the course indicated an increase in pass rates and 1-yr retention of students in the major. Now that time has passed and those students impacted by the revision have had time to graduate, this longitudinal study revisits and expands on these prior findings through examining student outcomes over a twelve year period (2001 through 2013). The student outcomes examined include failure rates in CS1, retention rates in the major, rates of switching into the major, time to degree, and performance in subsequent major courses. We compare these findings against similar metrics collected for another CS1 course at the same institution that caters to students with prior programming experience and did not make changes during this same time period. Overall, the inclusion of media computation, pair programming, and peer instruction corresponds to a significant improvement in passing rates for CS1 as well as retention of majors from CS1 through graduation. In turn, there is no indication that this larger group of students experienced any harm in terms of lower grades in upper-division courses or their time to degree.

Given the long-standing concern about students failing introductory programming courses, there is a need for interventions that may aid those students. In this work, we examine the potential benefit of three interventions based on prior computing education research (CER) or STEM education research literature: mindset interventions, the use of "Thinkathons" as an alternative to programming labs, and metacognitive interventions to encourage more productive study habits. We conducted an in-class study that controlled for both time-on-task and selection bias to investigate the potential benefits of integrating these interventions into the existing footprint of an introductory computing course. Despite the previously reported promise of the interventions we implemented, our findings were that in this context these techniques had only a mild positive effect for some students. We discuss possible reasons why these techniques are less successful than instructors might hope and argue for the need for more research on this topic.

The 32 chapters of the 2019 Cambridge Handbook of Computing Education Research synthesize the existing research in computing education and propose new directions for future research. An author from each chapter will summarize their chapter with auto-advancing slides. Attendees will be introduced to the breadth of content in the new handbook and can identify chapters of interest. This fits uniquely as a special session, and will likely be informative, inspiring, and overwhelming.

In this workshop participants will receive an overview of teaching practices in computer science that research indicates are effective. While the field of computer science education is young, it has uncovered several teaching practices that can be adopted by instructors that can improve both the retention and performance of students. These evidence based teaching practices include active learning techniques such as peer instruction and prior-knowledge activities, pair programming, and use of subgoal labels. Participants will experience firsthand many of these techniques and will be provided with resources on where to find more information, including the original research papers, on each technique. If you want to attend a workshop that will have an immediate impact in your class -- attend this one. The workshop will be interactive, engaging, and show you how to incorporate teaching practices that are empirically proven to provide benefits. You are guaranteed to leave with a list of many freely available resources and ideas to use in your next class. You will also have the opportunity to "ask the experts" as the authors of many of these research papers will be leading that session of the workshop.

Peer Instruction (PI) is a student-centric pedagogy in which students move from the role of passive listeners to active participants in the classroom. Over the past five years, there have been a number of research articles regarding the value of PI in computer science. The present work adds to this body of knowledge by examining outcomes from seven introductory programming instructors: three novices to PI and four with a range of PI experience. Through common measurements of student perceptions, we provide evidence that introductory computing instructors can successfully implement PI in their classrooms. We find encouraging minimum (74%) and average (92%) levels of success as measured through student valuation of PI for their learning. This work also documents and hypothesizes reasons for comparatively poor survey results in one course, highlighting the importance of the choice of grading policy (participation vs. correctness) for new PI adopters.

Although studies have shown Peer Instruction (PI) in computing courses to be beneficial for learning and retention, study of the student experience has been limited to attitudinal survey results. This study provides a preliminary evaluation of student experiences in a PI course -- specifically asking them to reflect on their role as a student in a PI lecture compared to a standard university lecture. Student responses to this question are first analyzed using Chi's Interactive-Constructive-Active-Passive framework which categorizes student activities by their value in a constructivist learning framework. This analysis finds that the majority of students reported activity in a PI lecture as "interactive" in contrast with "active" (e.g. taking notes) in a standard lecture. Additionally, a grounded theory open-coding analysis provides an initial examination of student perceptions of the PI lecture experience. Although students positively value learning-related aspects (feedback and increased understanding) a surprising breadth of value was noted around issues of affect and increased sense of community. In particular, these experiences invite discussion about PI and issues of STEM retention in post-secondary education.

This session will demonstrate best practices for integrating a variety of student engagement pedagogies into the classroom. Presenters play the role of instructors on the first day of class. Audience members play the role of students as they collaborate on a variety of sample activities. By providing models of active learning in computing contexts, we seek to motivate instructors to adopt these pedagogies in their classrooms. Attendees will be invited to join a growing community of faculty who share resources and best practices. Sufficient time will be reserved for questions and discussion.

Peer Instruction (PI) is a teaching method that supports student-centric classrooms, where students construct their own understanding through a structured approach featuring questions with peer discussions. PI has been shown to increase learning in STEM disciplines such as physics and biology. In this report we look at another indicator of student success the rate at which students pass the course or, conversely, the rate at which they fail. Evaluating 10 years of instruction of 4 different courses spanning 16 PI course instances, we find that adoption of the PI methodology in the classroom reduces fail rates by a per-course average of 61% (20% reduced to 7%) compared to standard instruction (SI). Moreover, we also find statistically significant improvements within-instructor. For the same instructor teaching the same course, we find PI decreases the fail rate, on average, by 67% (from 23% to 8%) compared to SI. As an in-situ study, we discuss the various threats to the validity of this work and consider implications of wide-spread adoption of PI in computing programs.

Beginning in 2008, we introduced a new CS1 incorporating a trio of best practices intended to improve the quality of the course, appeal to a broader student body, and, hopefully, improve retention in the major. This trio included Media Computation, Pair Programming, and Peer Instruction. After 3 and 1/2 years (8 CS1 classes, 3 different instructors, and 1011 students passing the course) we find that 89% of the majors who pass the course are still studying computing one year later. This is an improvement of 18% over our average retention of 71% for the previous version of the course (measured since Fall 2001). If the focus shifts from retention of passing CS1 majors to retention of CS1 initially enrolled majors, multiple improvements--fewer students drop, more students pass, and more passing students are retained--compound to increase retention by 31% (from 51% to 82%). In this paper we analyze further aspects of these results, detail the three instructional design choices, and consider how they impact issues known to affect retention.

In this paper we look at the impact on student learning of how a class is taught. We compare 2 sections of a non-majors CS0 course offered in the same term, by the same instructor, covering the same content and utilizing the same book, labs and exams. One section was taught using standard lecture practices including lecture from slides, live coding and weekly quizzes. The other section was taught using the Peer Instruction (PI) method that actively engages students in constructing their own learning, instead of absorbing understanding from the instructor's explanations. Using a factorial analysis of variance, we find a main effect of instructional method on final exam grade (F (1,200) = 5.87, p = 0.016) with students in the Peer Instruction section scoring an average 5.7% higher than in the standard lecture practices section. We find no significant interactions among gender and grade or class status (lower or upper division) and grade. In a separate analysis, we also find the interaction of instructional method and high school background to be significant (F (1,147) = 7.48, p = 0.007). In discussion we consider the meaning of these results for educators and describe questions for future work.

We introduce participants to Peer Instruction (PI): an active learning technique shown to be effective across the CS curriculum. In PI, Students work together to exchange perspectives and answer challenging conceptual questions, and are supported by short teaching segments. We will introduce and motivate PI, demonstrate its use in combination with a clicker system, and describe ways to encourage student preparation for PI classes. Pre-lecture preparation may include reading quizzes or exploratory homeworks, two topics of recent interest in the computing research literature. We will explore this literature as part of our tour of a complete PI course: from pre-lecture, to lecture, to the course at large.

We report on a post-hoc analysis of introductory programming lecture materials. The purpose of this analysis is to identify what knowledge and skills we are asking students to acquire, as situated in the activity, tools, and culture of what programmers do and how they think. The specific materials analyzed are the 133 Peer Instruction questions used in lecture to support cognitive apprenticeship -- honoring the situated nature of knowledge. We propose an Abstraction Transition Taxonomy for classifying the kinds of knowing and practices we engage students in as we seek to apprentice them into the programming world. We find students are asked to answer questions expressed using three levels of abstraction: English, CS Speak, and Code. Moreover, many questions involve asking students to transition between levels of abstraction within the context of a computational problem. Finally, by applying our taxonomy in classifying a range of introductory programming exams, we find that summative assessments (including our own) tend to emphasize a small range of the skills fostered in students during the formative/apprenticeship phase.

Constructivist learning theory suggests that learners must construct their own understandings, rather than have understanding passively dumped into their brains. These findings support the US National Research Council's recommendations for the adoption of active learning pedagogies in the classroom. However, the "classroom lecture" is only one of the resources students commonly have for learning in higher education. In this paper, we present exploratory homeworks -- a tool to support active learning for teaching programming languages. By leveraging the opportunity for the student to interact with the computer/compiler, we seek to provide a model for students of how to explore and understand programming language constructs and concepts. We report on the use of 15 exploratory homework assignments used in a CS0 course with 440 students in Winter 2011. We provide a model and advice for others wishing to develop exploratory homeworks for their programming courses and present quantitative and qualitative evidence regarding students' positive valuation of the homeworks.

We introduce participants to Peer Instruction (PI): an active learning technique applicable to the teaching of many subjects, including CS. In PI, Students work together to exchange perspectives and answer challenging conceptual questions, and are supported by short teaching segments. We will introduce and motivate PI, demonstrate its use in combination with a clicker system, and show that PI is much more than the use of clickers. Participants will work in groups to develop new PI questions addressing challenges to their students' learning, and discuss numerous pedagogical benefits conferred through PI.

Computing and computation are increasingly pervading our lives, careers, and societies - a change driving interest in computing education at the secondary level. But what should define a "general education" computing course at this level? That is, what would you want every person to know, assuming they never take another computing course? We identify possible outcomes for such a course through the experience of designing and implementing a general education university course utilizing best-practice pedagogies. Though we nominally taught programming, the design of the course led students to report gaining core, transferable skills and the confidence to employ them in their future. We discuss how various aspects of the course likely contributed to these gains. Finally, we encourage the community to embrace the challenge of teaching general education computing in contrast to and in conjunction with existing curricula designed primarily to interest students in the field.

PeerWise (PW) is an online tool that allows students in a course to collaborate and learn by creating, sharing, answering and discussing multiple-choice questions (MCQs). Previous studies of PW at the introductory level have shown that students in computing courses like it, and report statistically significant learning gains in courses taught by the investigators at different institutions. However, we recently conducted three quasi-experimental studies of PW use in upper-division computing courses in the U.S. and failed to replicate these positive results. In this paper we consider various factors that may impact the effectiveness of PW, including instructor engagement, usage requirements and subject-matter issues. We also report several positive results from other STEM courses at the same institution, discuss methodological issues pertaining to our recent studies and propose approaches for further investigation.

This paper discusses the results of a Grounded Theory study on students experience with introductory programming assignments in the light of social cognitive theory. In previous studies we have found CS majors experienced the process of doing CS1 programming assignments in different ways; but they universally made programming-related self-efficacy assessments along the way. Notably, students may reflect negatively on their self-efficacy after successfully completing an assignment, or positively after struggling with an assignment. CS majors tended to use their comparisons with self and classmates as a base for their self-efficacy perceptions. This paper takes a deeper look at these results from the lens of Bandura's self-efficacy theory with the goal of detailing viable pedagogical interventions to support students' introductory programming course experiences.

Peer Instruction (PI) is an instructional approach that engages students in constructing their own understanding of concepts. Students individually respond to a question, discuss with peers, and respond to the same question again. In general, the peer discussion portion of PI leads to an increase in the number of students answering a question correctly. But are these students really learning, or are they just "copying" the right answer from someone in their group? In an article in the journal Science, Smith et al. affirm that genetics students individually learn from discussion: having discussed a first question with their peers, students are better able to correctly, individually answer a second, conceptually-related question. We replicate their study, finding that students in upper-division computing courses (architecture and theory of computation) also learn from peer discussions, and explore differences between our results and those of Smith et al. Our work reveals that using raw percentage gains between paired questions may not fully illuminate the value of peer discussion. We define a new metric, Weighted Learning Gain, which better reflects the learning value of discussion. By applying this metric to both genetics and computing courses, we consistently find that 85-89% of "potential learners" benefit from peer discussion.

Peer Instruction (PI) has a significant following in physics, biology, and chemistry education. Although many CS educators are aware of PI as a pedagogy, the adoption rate in CS is low. This paper reports on four instructors with varying motivations and course contexts and the value they found in adopting PI. Although there are many documented benefits of PI for students (e.g. increased learning), here we describe the experience of the instructor by looking in detail at one particular question they posed in class. Through discussion of the instructors' experiences in their classrooms, we support educators in consideration of whether they would like to have similar classroom experiences. Our primary findings show instructors appreciate that PI assists students in addressing course concepts at a deep level, assists instructors in dynamically adapting their class to address student misunderstandings and, overall, that PI encourages students to be engaged in conversations which help build technical communication skills. We propose that using PI to engage students in these activities can effectively support training in analysis and teamwork skills.

Since 2008, NSF and The College Board, have been developing a "Computer Science: Principles" curriculum to "introduce students to the central ideas of computing and CS, to instill ideas and practices of computational thinking, and to have students engage in activities that show how computing and CS change the world". We report on the initial pilot of the CS Principles curriculum at 5 universities in 2010/11. The instructors from the pilot schools will describe their classes, the piloting experience (teaching under a microscope), and successes and failures. Emphasis will be on: mapping the CS Principles curriculum to a college's specific needs, and how others can use or modify the existing materials for pilots at their schools.

Part of computer science education research focuses on the design of new technologies and techniques for improving educational experiences. However, for these systems and techniques to be truly impactful, their use must be disseminated to the larger instructor population. Single-instance workshops, where instructors are brought together to learn about a new technique or system so that they can possibly adopt it, are a common dissemination method. Unfortunately, rarely do we see reports regarding their effectiveness. In this case study, we report on two NSF-funded single-instance workshops designed to support adoption of the Ubiquitous Presenter (UP) active learning classroom presentation system. Though only 44% of workshop attendees used the system in their classrooms, 65% of those used the system repeatedly. Overall this impacted 1570 students. Additionally, 60% of our attendees used the active learning features of UP - a much higher usage rate than in the general UP user population. We reflect on the aspects of the workshops which seemed to promote, and hinder, instructor adoption and, finally, suggest some metrics for evaluating innovation dissemination workshops in general.

Expertise is developed through both a) self-reflection and b) making useful plans for improvement [3, 10]. Traditional novice-level programming assignments require neither of these skills to be used. Could we get students to think about improving their software development processes? What areas would they identify as needing improvement? Could they write effective plans for themselves? In this experience report, we analyze the results of an intervention with 236 CS1.5 students asking them to do these activities. We find that they most commonly make improvements in planning, compared to coding and testing. Additionally, over half of the plans they make are so vague as to be of little use in helping students identify if they have, in fact, improved. Finally, we asked students at the end of the term to reflect on how their experiences with programming assignments changed over the term. We discuss our results in light of how instructors can focus instruction to help students become more meta-cognitive about their own software development processes.

The constant comparative method was used to explicate the experience computing majors have with programming assignments in a CS1 course. Findings from a series of four interviews conducted with a purposeful sample of nine students revealed that the primary reflective experience with programming assignments was emotional. That is, the way students remembered and discussed their experiences with programming assignments was dominated by emotional experiences and reactions. We identified six stages whose dimensions capture the variation in students' emotional experiences with programming assignments. This paper reports the beginnings of analysis geared at developing a theory of students' programming assignment experience, rich in detail and grounded in student experience. When completed such a theory may lead to curricular supports, interventions, or tools, to help steer student experience away from the most harmful of emotional tolls.

Is there consensus on what students should learn in CS2? Should they learn to use data structures, understand their specific implementation details, or both? Finally, has the computing education community's answer to the second question changed over time? In this paper, we begin to explore these questions based on an analysis of a key artifact instructors use to assess their students' performance: their final exams. Specifically, we look at two CS2 concepts as covered in those exams: stacks and hashtables.

Previous reports of a media computation approach to teaching programming have either focused on pre-CS1 courses or courses for non-majors. We report the adoption of a media computation context in a majors' CS1 course at a large, selective R1 institution in the U.S. The main goal was to increase retention of majors, but do so by replacing the traditional CS1 course directly (fully preparing students for the subsequent course). In this paper we provide an experience report for instructors interested in this approach. We compare a traditional CS1 with a media computation CS1 in terms of desired student competencies (analyzed via programming assignments and exams) and find the media computation approach to focus more on problem solving and less on language issues. In comparing student success (analyzed via pass rates and retention rates one year later) we find pass rates to be statistically significantly higher with media computation both for majors and for the class as a whole. We give examples of media computation exam questions and programming assignments and share student and instructor experiences including advice for the new instructor.

PeerWise is a collaborative web-based system that engages students in the creation and evaluation of a test bank of multiple-choice questions. Previous studies involving two large CS1 courses in New Zealand have provided preliminary evidence that PeerWise usage is positively correlated with exam performance. In addition, it was reported that the student generated assessment questions were mostly free from errors and were clearly written, and the students appeared to positively value the system. Here we report on the first use of PeerWise in a CS1.5 course (second programming course) in the United States. Although the usage model was modified slightly to accommodate pedagogical factors, we highlight similar positive outcomes to those observed in New Zealand. Of particular note, students who were most active using PeerWise improved their rank in class relative to their peers who were less active.

The SIGCSE community has been exploring the role of multimedia to enhance computing education since the earliest algorithm visualization systems and studies [1]. Media Computation is a shift in focus [2]. Where algorithm visualization presents information to the student to facilitate their understanding, media computation is about having students manipulate media as the data for their programming, i.e., as the focus of the course activities. Students in media computation produce new images, sounds, and video. We aim to show that computer science is about more than numbers and strings. Computer science is also about creative expression. The original media computation work focused on using media to motivate non-computing majors [2]. The role of media in motivating student learning for computing education has broadened. Inventive teachers are using media computation for lots of different kinds of students, at different kinds of institutions, with a range of languages and toolkits. This special session is a mixture of "Five Minute Madness," science fair, and art gallery. Each participant will present how he or she is using media to motivate student learning, and some student work will be available for audience inspection

Peer Instruction (PI) is a pedagogical technique to increase engagement in lectures. Students answer a multiple choice question (MCQ) typically using clickers (hand-held remote devices with a minimum of 5 option buttons), discuss the question with their peers, and then answer the question again. In physics, PI has years of evidence of increased conceptual learning, as measured by the Force Concept Inventory (FCI)[7]. In this experience report, we describe how PI was applied in CS1 and CS1.5 courses teaching Java. We identify specifics of the standard PI model which were adopted, adapted, or discarded for use in introductory computing, describe the process involved for the instructor, give examples of the types of questions asked of students, report on students' performance in answering these questions, reflect on the value for the instructor, and report the attitudes and opinions of the students. We conclude with observations, advice and suggested improvements.

Academic integrity and cheating are issues of specific importance in computing courses due to the restricted nature of much of our assigned work. Additionally, use of valued pedagogical and professional practices such as pair programming can muddy the waters when it comes to students' understandings and experiences with collaboration. In this study we report on 112 students at the beginning of a second programming course being asked to describe a scenario of student-to-student collaboration that "crosses the line" in terms of what should be allowed in the course. We find that students describe inappropriate acts involving sharing of code and sharing of information, with the former being more prevalent. Additionally, about half of the scenarios include mitigating circumstances that should not affect the propriety of those acts. Finally, when presented with other students' (often vague) scenarios, students have little consensus on whether those reflect appropriate or inappropriate collaborations.

This paper investigates what students understand about algorithm efficiency before receiving any formal instruction on the topic. We gave students a challenging search problem and two solutions, then asked them to identify the more efficient solution and to justify their choice. Many students did not use the standard worst-case analysis of algorithms; rather they chose other metrics, including average-case, better for more cases, better in all cases, one algorithm being more correct, and better for real-world scenarios. Students were much more likely to choose the correct algorithm when they were asked to trace the algorithms on specific examples; this was true even if they traced the algorithms incorrectly.

Fun, interesting, hard, rewarding, and challenging: these are the most frequent responses of 697 students from five institutions at the end of a first programming course. Student experience with introductory programming courses is of interest to the computing education community, especially due to continued decreases in enrollments in computing degree programs. In this study, we explore one direct approach to document students' initial attitudinal experiences with programming by asking them to complete an open-ended question at the end of a first programming course. Based on content-analysis of students' responses, we find that nearly 50% of responses were positive in nature, there is significant difference in the responses of majors and non-majors, and that response characteristics correlate to earned grade in the course. We present preliminary, but inconclusive evidence on the impact of context (e.g., gaming or media computation) in a first programming course. Finally, we propose a multiple-choice question based on the most common student responses for large-scale deployment in computing courses and identify key contextual information that will inform future analysis of that data.

We present what students say about their preferred learning style to succeed in introductory programming. Using the Felder-Silverman learning styles, students contrasted the 'best' learning style for programming with the 'best' for learning mathematics. Overall students believe that while they learn mathematics using a reflective style, learning to program is significantly more active. They also believe learning mathematics has a strong verbal component, whereas learning programming is primarily visual.

No abstract available.

We collected advice on how to succeed at learning to program from 164 CS1 students at 3 institutions during a "saying is believing" intervention designed to encourage a growth mindset. More students gave general advice (63%) than programming-specific (23%) or attitudinal advice (34%), despite being prompted to encourage future students to develop a growth mindset toward programming. Advice categories and quotes offer educators insights into student beliefs and practices and suggest a framework for considering how best to advise students. We discuss the implications of students offering advice to other students and provide a handout of representative advice intended for distribution to students in introductory programming courses.

Transitions from novice to expert often cause stress and anxiety and require specialized instruction and support to enact efficiently. While many studies have looked at novice computer science students, very little research has been conducted on professional novices. We conducted a two-month in-situ qualitative case study of new software developers in their first six months working at Microsoft. We shadowed them in all aspects of their jobs: coding, debugging, designing, and engaging with their team, and analyzed the types of tasks in which they engage. We can explain many of the behaviors revealed by our analyses if viewed through the lens of newcomer socialization from the field of organizational man-agement. This new perspective also enables us to better understand how current computer science pedagogy prepares students for jobs in the software industry. We consider the implications of this data and analysis for developing new processes for learning in both university and industrial settings to help accelerate the transition from novice to expert software developer.

Common exam practice centres around two question types: code tracing (reading) and code writing. It is commonly believed that code tracing is easier than code writing, but it seems obvious that different skills are needed for each. These problems also differ in their value on an exam. Pedagogically, code tracing on paper is an authentic task whereas code writing on paper is less so. Yet, few instructors are willing to forgo the code writing question on an exam. Is there another way, easier to grade, that captures the "problem solving through code creation process" we wish to examine? In this paper we propose Parson's puzzle-style problems for this purpose. We explore their potential both qualitatively, through interviews, and quantitatively through a set of CS1 exams. We find notable correlation between Parsons scores and code writing scores. We find low correlation between code writing and tracing and between Parsons and tracing. We also make the case that marks from a Parsons problem make clear what students don't know (specifically, in both syntax and logic) much less ambiguously than marks from a code writing problem. We make recommendations on the design of Parsons problems for the exam setting, discuss their potential uses and urge further investigations of Parsons problems for assessment of CS1 students.

Jane sees 50 compiler errors as a challenge. John sees them as defeat. Psychology research suggests these contrasting reactions may stem from students' self-theories, or their beliefs about themselves. Jane's reaction is characteristic of a growth mindset, the idea that with hard work and persistence, one's intelligence can increase. John's behavior is in line with a fixed mindset, the belief that individuals are born with a certain amount of intelligence and there is little they can do to change it. Numerous studies of self-theories have shown that students with a growth mindset perform better in academic settings; they cope more effectively with challenges, maintain higher grades, and are less susceptible to stereotype threat. In this study we attempted a "saying is believing" intervention to encourage CS1 students to adopt a growth mindset both in general and towards programming. Despite notable success of this type of intervention in a non-CS context, our results offered few statistically significant differences both from pre-survey to post-survey and between control and intervention groups. Further, the statistically significant results we did find differed in direction between institutions (some students exhibited more growth response, others less). We analyzed further evidence to explore possible confounding issues including whether our intervention even registered with students and how students interpreted the questions which we used to assess their self-theories.

This paper explores the programming knowledge of novices using Biggs' SOLO taxonomy. It builds on previous work of Lister et al. (2006) and addresses some of the criticisms of that work. The research was conducted by studying the exam scripts for 120 introductory programming students, in which three specific questions were analyzed using the SOLO taxonomy. The study reports the following four findings: when the instruction to students used by Lister et al. - "In plain English, explain what the following segment of Java code does" - is replaced with a less ambiguous instruction, many students still provide multistructural responses; students are relatively consistent in the SOLO level of their answers; student responses on SOLO reading tasks correlate positively with performance on writing tasks; postgraduates students manifest a higher level of thinking than undergraduates.

How do new college graduates experience their first software development jobs? In what ways are they prepared by their educational experiences, and in what ways do they struggle to be productive in their new positions? We report on a "fly-on-the-wall" observational study of eight recent college graduates in their first six months of a software development position at Microsoft Corporation. After a total of 85 hours of on-the-job observation, we report on the common abilities evidenced by new software developers including how to program, how to write design specifications, and evidence of persistence strategies for problem-solving. We also classify some of the common ways new software developers were observed getting stuck: communication, collaboration, technical, cognition, and orientation. We report on some common misconceptions of new developers which often frustrate them and hinder them in their jobs, and conclude with recommendations to align Computer Science curricula with the observed needs of new professional developers.

A qualitative analysis of debugging strategies of novice Java programmers is presented. The study involved 21 CS2 students from seven universities in the U.S. and U.K. Subjects "warmed up" by coding a solution to a typical introductory problem. This was followed by an exercise debugging a syntactically correct version with logic errors. Many novices found and fixed bugs using strategies such as tracing, commenting out code, diagnostic print statements and methodical testing. Some competently used online resources and debuggers. Students also used pattern matching to detect errors in code that "just didn't look right". However, some used few strategies, applied them ineffectively, or engaged in other unproductive behaviors. This led to poor performance, frustration for some, and occasionally the introduction of new bugs. Pedagogical implications and suggestions for future research are discussed.

Classroom note taking tends to be a private activity, hiding a wealth of knowledge in both content and method. With the advent of the web, whose technology and culture seemingly invites everyone to share everything, we are prompted to ask how making note taking a public activity -- noteblogging -- might advance learning. What does a blog about a computer science classroom look like? What supports are needed to enable noteblogging as a form of auxiliary instruction? In this paper we overview the design and use of noteblogging as part of the Ubiquitous Presenter digital classroom system. With NoteBlogger, students with Tablet PCs take handwritten notes digitally on top of the instructor's slides, and their notes are instantly reviewable by other students in class using a web browser. We examine the impact of noteblogging on a CS1 course through interviews with bloggers (to reveal their motivation and understand their choice of content) and blog watchers (to learn when and why they watch). We also analyze the blogs in this CS1 course to identify their educational contribution. We find that noteblogging enables a unique classroom participation model that specifically engages more advanced students. Blog content spans many levels of intellectual engagement, which can support a range of learners in CS1 as well as perhaps model for them various levels of reflection.

Pair programming, as part of the Agile Development process, has noted benefits in professional software development scenarios. These successes have led to a rise in use of pair programming in educational settings, particularly in Computer Science 1 (CS1). Specifically, McDowell et al. [2006] has shown that students using pair programming in CS1 do better in a CS2 class (with solo programming) than students who don't pair in CS1. This paper seeks to address a similar question, but from a qualitative, student-focused approach. How do students define, experience, and value the pair programming experience? How do they experience and value it compared to solo programming? Does pairing in CS1 impact their confidence in their abilities? We report on semi-structured interviews with 11 subjects from two institutions where pair programming was used in CS1, and solo programming was used in the CS2. Many of the responses met our expectations; students get stuck less and explore more ideas while pairing, and believe that pair programming helped them in CS1. Other responses were more surprising. Students reported that when solo programming they were more confident and understood their programs better. Many students also said that they started work on their assignments earlier when soloing. Students also continue to use other students as resources even when working "solo."

As the third in a series of projects investigating commonsense computing -- the relevant knowledge that students have before any formal study of computing -- we examine students' commonsense understanding of concurrency. Specifically, we replicated (with modifications) an experiment by Ben-David Kolikant. [2] Ben-David Kolikant's data were gathered from high-school seniors who had previously studied computing, at the beginning of an advanced class in concurrent and distributed programming. Modifying one of her questions to reflect our students' lack of background, we asked students at five different institutions, in the first week of CS1, to describe in English the problems that might arise when more than one person is selling seats to a concert.Almost all students (97%) identified the problem of interest -- that a race condition may occur between sellers. 73% of students identified at least one possible solution. We found that the categorizations developed by Ben-David Kolikant were also meaningful when applied to our data, that our beginning CS1 students are more likely to give centralized solutions (as opposed to decentralized ones) than Ben-David Kolikant's concurrency students, and that the granularity of solutions is finer among the more experienced students.

Pair programming, as part of the Agile Development process, has noted benefits in professional software development scenarios. These successes have led to a rise in use of pair programming in educational settings, particularly in CS1. Specifically [6] has shown that students using pair programming in CS1 do better in a CS2 class (with solo programming) than students who don't pair in CS1. This paper seeks to address a similar question, but from a qualitative, student-focused approach. How do students define, experience and value the pair programming experience? How do they experience and value it compared to solo programming? Does pairing in CS1 impact their confidence in their abilities.

We examine students' commonsense understanding of computer science concepts before they receive any formal instruction in the field. For this study, we asked students on the first day of a CS1 class to describe in English how they would arrange a set of numbers in ascending, sorted order; we then repeated the experiment asking students to sort a list of dates (in mm/dd/yyyy format).We found that a majority of students described a coherent algorithm; some described versions of insertion or selection sort, but many gave unexpected algorithms. We also found significant differences between responses given for sorting numbers versus dates. Based on our analysis of the data we suggest that beginning-programming instructors more explicitly discuss data types, begin loop instruction with post-test loops, assist students in recognizing implicit conditional and iteration use in natural language solutions to probls, and recognize that novices and experts focus on different aspects of the probl in even basic probl solving tasks.

Peer review is widely recognized for advancing student learning, in particular for developing reflective processes like critical thinking. The classroom is ripe for peer review because the subject matter is fresh and in-depth interactivity is possible. Yet the limited time available in class conflicts with peer review's deliberative nature. We hypothesize that peer review -- at least the initial stages of it -- can be supported in the classroom with tools for facilitating the rapid identification of interesting issues for discussion. The potential benefits of such a tool include: furthering the student-focus of in-class active learning activities, further implanting critical analysis skills through frequent in-class use, supporting immediate feedback, and enabling comparison of student and instructor-modeled critical analysis.This paper explores tool support for in-class lightweight preliminary peer-review (LPPR): peer review that is instigated in the classroom, but does not necessarily end there. We proposed that students classify peer solutions in 4 dimensions: correctness, comprehension (e.g., "do I understand this solution"), worthiness for discussion, and similarity to the evaluator's own solution. We designed an LPPR extension to Ubiquitous Presenter, and then conducted an exploratory study in a mock classroom setting. We found that LPPR can quickly identify a subset of student solutions that warrant immediate discussion, and that modest amounts of reflection arise from the LPPR process.

Research has shown that educational technology can broaden and enhance the use of active learning in large classrooms. An educational technology platform often relies on students to bring laptops or specialized wireless devices like clickers to interact through the system. Mobile phones are an attractive alternative, as most students already possess them, they have more capabilities than dedicated clickers, and yet are small enough to minimize interference with note taking on a classroom desk.This paper presents the design and use of a mobile phone extension to Ubiquitous Presenter, which allows students to submit solutions to active learning exercises in the form of text or photo messages. In an exploratory study, students found that text messaging worked well for exercises with multiple choice or short answers. Entering symbols common to computer science was difficult. Many problems were more suitable to photo messaging of a handwritten answer, although image quality must be managed. The phone's small size left space for the use of a notebook. The students had concerns about the message charges that would accrue in use. In conclusion, we offer recommendations to instructors and system designers interested in leveraging mobile phones to increase communication in the classroom.

We examine students' commonsense understanding of computer science concepts before they receive any formal instruction in the field. Specifically, we asked students on the first day of a CS1 class to describe in English how they would arrange a set of numbers in ascending, sorted order. We repeated the experiment with students in an introductory economics course, and again with a sub-population of the CS1 students after ten weeks of Java instruction.We found that a majority of beginning computing students could describe a coherent algorithm to correctly sort a list of numbers, while less than a third of general college students could do so. Many students gave versions of selection or insertion sort, but the most common algorithm treated numbers as strings and manipulated them digit by digit. Students who used iteration strongly preferred post-test loops. Finally, some aspects of student performance became worse after ten weeks of CS1 instruction.

We are studying what students know about computer science-related topics before they take formal coursework at the university level. Preliminary results suggest that entering students have a fairly sophisticated understanding of algorithms. We are exploring other central computing topics for similar shared commonsense understanding.

In this demonstration participants will experience the potential for fast, scalable active learning in the classroom using standard web technology. Ubiquitous Presenter (UP) enables students with web-enabled devices (computers, laptops, Tablet PCs, cell phones, PDAs, etc.) to interact in (or outside) class in a range of manners and through a modality of their choosing [2,3]. Their input (via a web browser) is immediately and anonymously transmitted to the instructor machine, where it can be reviewed and selected for display to the class via digital projector. The instructor, using a Tablet PC, can additionally comment on, augment, or correct student input real time in class. These activities, along with instructor designed lecture slides and in-class ink are automatically archived on a web server for review after class by both instructors and students.

This paper reports on the authors use of the SOLO taxonomy to describe differences in the way students and educators solve small code reading exercises. SOLO is a general educational taxonomy, and has not previously been applied to the study of how novice programmers manifest their understanding of code. Data was collected in the form of written and think-aloud responses from students (novices) and educators (experts), using exam questions. During analysis, the responses were mapped to the different levels of the SOLO taxonomy. From think-aloud responses, the authors found that educators tended to manifest a SOLO relational response on small reading problems, whereas students tended to manifest a multistructural response. These results are consistent with the literature on the psychology of programming, but the work in this paper extends on these findings by analyzing the design of exam questions.

What measurable effect do the language and paradigm used in early programming classes have on novice programmers' ability to do design? This work investigates the question by using data collected from 136 first competency students as part of a multi-national, multi-institutional study of students' approach to and attitudes toward design. Analysis of a number of surface characteristics of their designs found strikingly few differences between designs produced by students at schools that teach using objects-early, imperative-early, and functional-early paradigms. A similar lack of difference was found between students at C++-first and Java-first schools. While statistically significant differences are found for three characteristic comparisons across language and paradigm, these results seem to have little meaning for teaching given the complexity of the null hypotheses tested in those three cases. In particular, for the following design characteristics no statistically significant differences across language or paradigm of early instruction were found: attempt to address requirements, type of design produced, number of parts in design, recognition of ambiguity in design, and connectedness of design.

Experimentation has shown that in-class educational technologies, by permitting anonymous, authored participation, can dramatically alter student communications in the classroom. Now, the appearance of dual pen-and-keyboard computing devices in the university classroom, notably Tablet PCs, motivates thinking critically about how different expressive modalities could improve in-class student problem -solving and communication.This paper describes the use of Ubiquitous Presenter 2.0 in a study to discover the driving issues of multimodality for both in-class technologies and student exercises. This paper sensitizes instructors to the issues of modality and makes specific recommendations for application design. We find that the choice of modality is not merely one of efficiency or naturalness, but is loaded with numerous personal, social, and material considerations. Although use of the pen (over typed text) is generally preferred, we find that choice itself is critical to encouraging student creativity, collaboration, and communication.

How do beginning students approach problems which require them to read and understand code? We report on a Grounded Theory-based analysis of student transcripts from 12 institutions where students were asked to "think aloud" when solving such problems. We identify 19 strategies used by students. Primary results are that all students employ a range of strategies, there were (in total) many different strategies that were applied, students use multiple strategies on each individual problem, students applied different strategies to different types of questions, and students often applied strategies poorly. We show that strategies conform with existing education theories including Bloom's Taxonomy and the Approaches to Study Inventory. Additionally, we discuss emergent theories developed through a card sort process.

Successful software engineering requires experience and acknowledgment of complexity, including that which leads designers to recognize ambiguity within the software design description itself. We report on a study of 21 post-secondary institutions from the USA, UK, Sweden, and New Zealand. First competency and graduating students as well as educators were asked to perform a software design task. We found that as students go from first competency to graduating seniors they tend to recognize ambiguities in under-specified problems. Additionally, participants who recognized ambiguity addressed more requirements of the design.

The University of Washington's Classroom Presenter lecturing system enables an active lecturing environment by combining a standard electronic slide presentation format with the capability for extemporaneous ink annotations by instructors and students using Tablet PCs. Thus, it can promote more interactive, student-centered learning. While many students may own laptops, few are yet Tablet devices. Also, Presenter uses multicast networking, which has availability and reliability issues. Ubiquitous Presenter (UP) expands Presenter via common web technologies to support non-Tablet audiences and enhance student control. UP enables students, using internet web browsers, to (a) synchronously or asynchronously view the slides and ink that are broadcast by the instructor, as well as (b) provide contextual submissions via text overlaid on the instructor's slides. The only compromises are that non-Tablet students cannot produce ink, and that professor ink is provided after a small time delay.

There has been much research on the benefits of active and collaborative learning and on its use in computer science courses. As classroom technology becomes more prevalent it is natural to develop systems that support the use of these techniques. We have developed such a system as an extension to Classroom Presenter [2], a Tablet PC-based presentation system. In our system students (or groups of students) are equipped with tablet computers and at various points during the lecture, are asked to solve a problem or respond to a question. Students respond by writing their solution on the tablet and submitting it wirelessly to the instructor. The instructor can view all student responses, select one or more to display to the class, and annotate responses with ink as they are being displayed. Student responses can be saved for review after lecture by the instructor or made available electronically to the students. In this paper we describe the system and our initial experiences using the system in two classes (CS1 and Computer Ethics) at the University of San Diego in Fall 2003.

Computer science instructors frequently teach using slides displayed with a computer and a data projector. This has many advantages, e.g., ability to present prepared materials and ease of switching the display to a development environment during mid-presentation. However, existing computer-based presentation systems severely limit flexibility in delivery, hindering instructors' extemporaneous adaptation of their presentations to match their audiences. One major limitation of computer-based systems is lack of support for high-quality handwriting over slides, as with overhead projectors and other manual presentation systems. We developed and deployed Classroom Presenter, a Tablet PC-based presentation system that (1) combines the advantages of existing computer-based and manual presentation systems and (2) builds on these systems, introducing novel affordances. Classroom Presenter has been used in 25 Computer Science courses at three universities. In this paper we describe the system, summarize results from its deployment, and detail several novel uses of the system by instructors in computer science courses.

Predicated execution can be used to alleviate the costs associated with frequently mispredicted branches. This is accomplished by trading the cost of a mispredicted branch for execution of both paths following the conditional branch. In this paper we examine two enhancements for branch prediction in the presence of predicated code. Both of the techniques use recently calculated predicate definitions to provide a more intelligent branch prediction. The first branch predictor, called the squash false path filter, recognizes fetched branches known to be guarded with a false predicate and predicts them as not-taken with 100% accuracy. The second technique, called the predicate global update branch predictor, improves prediction by incorporating recent predicate information into the branch predictor. We use these techniques to aid the prediction of region-based branches. A region-based branch is a branch that is left in a predicated region of code. A region-based branch may be correlated with predicate definitions in the region in addition to those that define the branch's guarding predicate.

This paper studies the relationship between storage requirements and performance. Storage-related dependences inhibit optimizations for locality and parallelism. Techniques such as renaming and array expansion can eliminate all storage-related dependences, but do so at the expense of increased storage. This paper introduces the universal occupancy vector (UOV) for loops with a regular stencil of dependences. The UOV provides a schedule-independent storage reuse pattern that introduces no further dependences (other than those implied by true flow dependences). OV-mapped code requires less storage than full array expansion and only slightly more storage than schedule-dependent minimal storage.We show that determine if a vector is a UOV is NPcomplete. However, an easily constructed but possibly nonminimal UOV can be used. We also present a branch and bound algorithm which finds the minimal UOV, while still maintaining a legal UOV at all times.Our experimental results show that the use of OV-mapped storage, coupled with tiling for locality, achieves better performance than tiling after array expansion, and accommodates larger problem sizes than untilable, storage-optimized code. F'urthermore, storage mapping based on the UOV introduces negligible runtime overhead.