This microteaching session is like Nifty Assignments for instruction. Instead of having the presenters just talk about their teaching, they will simulate how they would actually teach something. Covering a range of topics and grade levels, six educators will demo how they would teach a specific topic. To help identify the pedagogical practices that cut across grade bands and topics, the moderator, Colleen Lewis, will describe how their pedagogical practices connect with education research. The goal of the session is to inspire SIGCSE attendees by highlighting innovative instruction by exceptional educators. Attendees can adopt the content and/or pedagogical practices from each microteaching example.

This paper reports on the longitudinal impacts of an inclusive, structured research experience program for early career undergraduates in computer science that engages a large number of students from minoritized groups. We compared academic performance and retention in the major for program participants at two large public research universities in the United States vs. a matched control group of demographically and academically similar students. We found that the retention rate of program participants was higher than the control at both universities, though not statistically significantly so. We found no significant difference in post-program GPA, and the program did not erase equity gaps in GPA by race and first generation status that existed before the program. These results help us understand the benefits and limitations of large-scale early research programs for increasing equity in computer science.

This session takes inspiration from the highly successful "Nifty Assignments" special session, but instead highlights high quality multiple-choice questions that can be used for Peer Instruction. Peer Instruction is a pedagogical practice characterized by asking students to answer challenging, conceptual questions in class. For each question, students individually respond, discuss the question in small groups, and respond again based on their new understanding. Peer Instruction has been widely identified as an important instructional technique in teaching computing. In this session, members of the community will present some of their best Peer Instruction questions along with a short explanation that provides the pedagogical content knowledge indicating why the question is a good question. If you are interested in learning more about Peer Instruction or finding new questions for your course(s), this session is for you.

A Concept Inventory (CI) is an assessment to measure student conceptual understanding of a particular topic. This article presents the results of a CI for basic data structures (BDSI) that has been previously shown to have strong evidence for validity. The goal of this work is to help researchers or instructors who administer the BDSI in their own courses to better understand their results. In support of this goal, we discuss our findings for each question of the CI using data gathered from 1,963 students across seven institutions.

Computer Science now has a number of validated instruments available for measuring student knowledge or interest in computing including the Second CS1 Assessment (SCS1), The Basic Data Structures Inventory (BDSI), the Computing Attitudes Survey (CAS), and the Digital Logic CI. These instruments can be used by instructors to assess their students and/or their own teaching. They can also be used by researchers to measure students' learning or attitudes. The goal of this BOF is to help instructors and researchers gain a better understanding of how to use these instruments, whether that be to get started in education research, to compare student learning across terms/curricular revisions, or just to learn more about student misconceptions. We will begin by discussing the available instruments, their purpose, and how to obtain them. Then we will open the discussion to the group on what they would like to measure, how these instruments might work for them, and how to best employ them with their students.

The goal of the session is to help attendees who are committed to diversity and inclusion learn to talk about different dimensions of identity (e.g., race, class, gender, sex, sexuality, etc.). The landscape of terms is always changing and we want SIGCSE attendees to feel more comfortable using current language to talk about issues related to diversity and inclusion. This special session will include eight short videos, individual reflection, and a Q&A with presenters. This is an expansion of a special session held at the NCWIT Summit in 2019; it was well received and we hope to offer it to the larger SIGCSE audience. The format also lends itself to attendees sharing the videos and discussions with their colleagues after SIGCSE, which are available at http://ncwit.org/intersectionality-videos

Computer Science now has a number of validated instruments available for measuring student knowledge or interest in computing (SCS1, BDSI, CAS, Digital Logic CI, etc.). But when and how should instructors and researchers use these instruments? In this BOF, we will begin by discussing the available instruments, their purpose, and how to obtain them. Then we will open the discussion to the group on what they would like to measure, how these instruments might work for them, and how to best employ them with their students.

Undergraduates have been an important part of the teaching staff at many universities for decades. Recent work such as the Peer Teaching Summit at SIGCSE 2019 [1] and a systematic literature review [2] have focused more attention on issues related to the use of undergraduates in teaching assistant roles. This BOF provides a forum to discuss open research questions about undergrad TA/mentor programs at various stages in their evolution. Attendees will have an opportunity to discuss research questions, research methods, and explore possible collaborations. Discussion Leader(s): Diba Mirza will open the session by summarizing the key findings of literature review on UTAs from ICER'19 [1]-in particular, the fact that while there is widespread consensus that using UTAs is a good idea, the evidence backing up this consensus is mostly anecdotal. This creates many opportunities to establish the effectiveness of current practices, and the claimed benefits of the use of UTAs through more rigorous research and to discuss innovative ways to incorporate UTAs in teaching outside of what has been reported in the literature. This will be followed by a themed discussion to brainstorm about research within each of these areas. Based on the large turnout at the BOF on Undergraduate Teaching Assistants in SIGCSE'19, we plan to organize the discussion in smaller groups. Diba Mirza, Phillip Conrad, and Cynthia Lee will lead the discussion within each group. The leads will also document the discussions and share it with the participants.

Concept inventories (CIs) allow researchers and practitioners to measure student conceptual learning within a course or topic area. While they have enabled meaningful pedagogical change in other disciplines, there are relatively few CIs in computer science. In this paper, we report on our experiences as recent developers of a CI for basic data structures. We discuss each step along the route to a CI and offer tips based on what we have learned. We encourage others to create CIs, and we hope that this paper will serve as a practical guide through the process.

A Concept Inventory (CI) is a validated assessment to measure student conceptual understanding of a particular topic. This work presents a CI for Basic Data Structures (BDSI) and the process by which the CI was designed and validated. We discuss: 1) the collection of faculty opinions from diverse institutions on what belongs on the instrument, 2) a series of interviews with students to identify their conceptions and misconceptions of the content, 3) an iterative design process of developing draft questions, conducting interviews with students to ensure the questions on the instrument are interpreted properly, and collecting faculty feedback on the questions themselves, and 4) a statistical evaluation of final versions of the instrument to ensure its internal validity. We also provide initial results from pilot runs of the CI.

Peer instruction pedagogy is a student-centric approach that encourages students to read lecture material before coming to class and engages them in class via group discussion and preplanned conceptual questions. Peer instruction has shown promising results in core computer science courses such as Theory of Computation and Computer Architecture, as well as reducing failure rates and improving student retention in computer science major. This paper presents the results of the first-ever attempt to replicate these results in a cybersecurity course, using an action research methodology to implement and evaluate peer instruction in a semester-long cybersecurity course, Introduction to Computer Security. The evaluation consists of quizzes, subjective exams, peer instruction questions, and attitudinal surveys gathered over two control semesters and one peer instruction condition semester. We find evidence of learning gains during group discussion and improvements in dropout and failure rates compared to traditional lecture classes. In attitudinal surveys, most students report that they would recommend that other instructors use peer instruction.

Undergraduates have been an important part of the teaching staff at many universities for decades, but this is not a universal practice. This BOF provides a forum to discuss undergraduate TA/mentor programs at various stages in their evolution. Attendees will have an opportunity to discuss the benefits, best practices, and research questions related to the use of undergraduates as teaching assistants and/or mentors. The audience is expected to consist of faculty that have already implemented such programs, and those who might be considering doing so, and want to share information about successes and challenges. Discussion Leader(s): Phill Conrad will facilitate the discussion and invite Colleen Lewis, Cynthia Lee and Diba Mirza to briefly speak about the Undergraduate TA/Mentor programs at their respective institutions. During the balance of the time participants will be invited to contribute ideas or ask questions related to the use of undergraduate TAs and mentors. Expertise of Discussion Leader(s): Diba Mirza and Phill Conrad are faculty in the Computer Science department at UC Santa Barbara, where they have created an undergraduate mentor program in the past two years. Colleen Lewis is an Associate Professor of CS at Harvey Mudd College, and is the lead PI for the csteachingtips.org project, a project to collect and document CS pedagogical content knowledge. Cynthia Lee is a Lecturer in CS at Stanford University, and works closely with the longstanding undergraduate TA program in the department.

To be effective instructors and CS education researchers, we must identify and understand student difficulties surrounding core computing topics. This study examines student difficulties with the basic data structures commonly found in CS2 courses. Initial exploration of student thinking began with think-aloud interviews with students. These interviews centered on open-ended questions that were iteratively improved upon based on analysis of interview transcripts. The revised open-ended questions were then posed to 249 students during an end-of-term final exam study session. Using the explanations and justifications included by students, responses to the questions were coded and summarized. This work characterizes the difficulties revealed by student responses, and provides details of their prevalence among the examined student population.

Meaningful and engaging assignments are important to retention in CS. An interesting problem context may be able to make routine practice of programming basics more interesting for students. Problem contexts also provide the opportunity to bring in content related to social justice topics, which are important for providing students a well-rounded education. With funding from the NSF (#1339404), we have developed eight homework assignments that integrate social justice topics as the problem context for CS1 assignments. Workshop attendees will work in small groups to revise or adapt existing assignments, translate existing assignments into the language of their course, or develop a new assignment. Attendees will be encouraged to submit their work to Nifty Assignments for 2019 and NCWIT's peer-reviewed curriculum repository, Engage CS Edu (engage-csedu.org). All assignments will be posted on CSTeachingTips.org to be shared with the community.

Establishing learning goals for a course allows instructors to design course content to address those goals, helps students to focus their learning appropriately, and enables researchers to assess learning of those goals. In this work, we propose six learning goals for a topic prevalent in CS2 courses: Basic Data Structures. These learning goals arise from reviewing several CS2 courses at a variety of institutions, surveying faculty experts who commonly teach CS2, and meeting and working closely with these experts. We outline our process for creating learning goals, identify important topics underlying these goals, and provide examples of how the goals developed on the path to consensus. We also document that the term "CS2" does not have a unified interpretation within the CS education community and describe how this hurdle influenced our decision to focus on Basic Data Structures.

As classes grow beyond hundreds or even a thousand students, instructors must come to grapple with handling course logistics at scale. Otherwise mundane tasks can no longer be handled in an ad hoc way, and standard course design challenges become more acute. Just to name a few issues, the instructor might have to contend with: Providing timely effective feedback to students, exam scheduling, recruiting and managing a teaching staff that consists of dozens of people, ensuring consistency of grading, identifying and providing interventions for struggling students, providing a consistent policy for makeup work, and creating many types of course content (e.g. lab, HW, discussion section) that allows for an efficient path to mastery for each student regardless of diverse backgrounds and ability. If you have good ideas for handling these issues you'd like to share, or just want to learn what others are doing, come join us! Discussion will ideally include staffing techniques, technologies (including automated assessments), and ways of developing course content. A master list of such tips, as well as a retrospective summary, will be compiled and provided to the SIGCSE community at http://tinyurl.com/wotb2017.

In this workshop participants will receive an overview of teaching practices in computer science that research indicates are effective. While the field of computer science education is young, it has uncovered several teaching practices that can be adopted by instructors that can improve both the retention and performance of students. These evidence based teaching practices include active learning techniques such as peer instruction and prior-knowledge activities, pair programming, and use of subgoal labels. Participants will experience firsthand many of these techniques and will be provided with resources on where to find more information, including the original research papers, on each technique. If you want to attend a workshop that will have an immediate impact in your class -- attend this one. The workshop will be interactive, engaging, and show you how to incorporate teaching practices that are empirically proven to provide benefits. You are guaranteed to leave with a list of many freely available resources and ideas to use in your next class. You will also have the opportunity to "ask the experts" as the authors of many of these research papers will be leading that session of the workshop.

Peer Instruction (PI) is a student-centric pedagogy in which students move from the role of passive listeners to active participants in the classroom. Over the past five years, there have been a number of research articles regarding the value of PI in computer science. The present work adds to this body of knowledge by examining outcomes from seven introductory programming instructors: three novices to PI and four with a range of PI experience. Through common measurements of student perceptions, we provide evidence that introductory computing instructors can successfully implement PI in their classrooms. We find encouraging minimum (74%) and average (92%) levels of success as measured through student valuation of PI for their learning. This work also documents and hypothesizes reasons for comparatively poor survey results in one course, highlighting the importance of the choice of grading policy (participation vs. correctness) for new PI adopters.

Recent work in computing has converged on a collection of complementary findings suggesting the value of the Peer Instruction (PI) pedagogy. Compared to lecture, PI has been shown to decrease fail rates, increase final exam grades, and increase engagement and enjoyment. In PI, students work together to exchange perspectives and use clickers to answer challenging conceptual questions in the presence of a knowledgeable instructor.

Open educational resources - assignments, labs, course notes, and other types of materials made available for anyone interested in using them - have the potential to have a significant positive impact on courses and students at many institutions. Distribution and use of course materials is also an important factor in encouraging the adoption and use of innovative teaching practices. However, the use of open resources is still somewhat uncommon, with most instructors preferring to use their own materials.

Recent work in computing has converged on a collection of complementary findings suggesting the value of the Peer Instruction (PI) pedagogy. Compared to lecture, PI has been shown to decrease fail rates, increase final exam grades, and increase engagement and enjoyment. In PI, students work together to exchange perspectives and use clickers to answer challenging conceptual questions in the presence of a knowledgeable instructor.

New CS1 curricula and pedagogies have resulted in many positive outcomes over the last several years including lower fail rates and increased long-term retention. Given these positive outcomes, the question becomes how much do the traditional factors of prior experience and confidence still play a role in students' performance in and attitudes about these courses' Furthermore, given that increasingly recommended collaborative pedagogies (e.g. pair programming) force students to interact with their peers for a large percentage of their work in the class, how much does the confidence of their peers affect their own attitudes and performance? This paper presents a study investigating these questions. We find that prior experience and confidence still predict success, but only for some students. We also find that student confidence levels have little to no impact on the attitudes and performance of their peers.

As computer programming is increasingly considered an essential literacy skill for all students, MATLAB courses in particular can play a role in introducing non-major students to a tool commonly used in many of their fields. This paper reports on our department's experience introducing a CS1 in MATLAB for non-majors course. The course assumed no prior programming experience and no training in linear algebra. Without linear algebra and without the ability to do domain-specific tailoring, we turned to Media Computation to contextualize the skills and motivate students. Media Computation is an approach to programming instruction that focuses on manipulation of visual, audio, and video media. The course design also featured the Peer Instruction lecture format, in which lectures are punctuated by frequent questions that students answer individually and in small groups. To our knowledge, this represents the first time that Media Computation and Peer Instruction pedagogies have been comprehensively adapted to a MATLAB course. This work shares selected materials designed for this course, and reports outcomes of the two terms the course has been offered.

Peer Instruction (PI) is a teaching method that supports student-centric classrooms, where students construct their own understanding through a structured approach featuring questions with peer discussions. PI has been shown to increase learning in STEM disciplines such as physics and biology. In this report we look at another indicator of student success the rate at which students pass the course or, conversely, the rate at which they fail. Evaluating 10 years of instruction of 4 different courses spanning 16 PI course instances, we find that adoption of the PI methodology in the classroom reduces fail rates by a per-course average of 61% (20% reduced to 7%) compared to standard instruction (SI). Moreover, we also find statistically significant improvements within-instructor. For the same instructor teaching the same course, we find PI decreases the fail rate, on average, by 67% (from 23% to 8%) compared to SI. As an in-situ study, we discuss the various threats to the validity of this work and consider implications of wide-spread adoption of PI in computing programs.

We introduce participants to Peer Instruction (PI): an active learning technique shown to be effective across the CS curriculum. In PI, Students work together to exchange perspectives and answer challenging conceptual questions, and are supported by short teaching segments. We will introduce and motivate PI, demonstrate its use in combination with a clicker system, and describe ways to encourage student preparation for PI classes. Pre-lecture preparation may include reading quizzes or exploratory homeworks, two topics of recent interest in the computing research literature. We will explore this literature as part of our tour of a complete PI course: from pre-lecture, to lecture, to the course at large.

Peer Instruction has recently gained interest in computing as an effective active learning pedagogy. The general focus of PI research has been on the in-class portion of PI: multiple choice questions and group discussion. Here, our focus is the reading quizzes completed by students for purposes of class preparation. These quizzes contain content questions but also ask for difficulties or confusion with course material. Consistent with expectations, we demonstrate that providing correct responses to quiz questions positively correlates with other course assessments. Somewhat counter-intuitively, we find that identifying confusions, noting problematic sections, or asking questions about the reading are also correlated with lab grades.

Peer Instruction (PI) is an active learning pedagogical technique. PI lectures present students with a series of multiple-choice questions, which they respond to both individually and in groups. PI has been widely successful in the physical sciences and, recently, has been successfully adopted by computer science instructors in lower-division, introductory courses. In this work, we challenge readers to consider PI for their upper-division courses as well. We present a PI curriculum for two upper-division computer science courses: Computer Architecture and Theory of Computation. These courses exemplify several perceived challenges to the adoption of PI in upper-division courses, including: exploration of abstract ideas, development of high-level judgment of engineering design trade-offs, and exercising advanced mathematical sophistication. This work includes selected course materials illustrating how these challenges are overcome, learning gains results comparing these upper-division courses with previous lower-division results in the literature, student attitudinal survey results (N = 501), and pragmatic advice to prospective developers and adopters. We present three main findings. First, we find that these upper-division courses achieved student learning gains equivalent to those reported in successful lower-division computing courses. Second, we find that student feedback for each class was overwhelmingly positive, with 88% of students recommending PI for use in other computer science classes. Third, we find that instructors adopting the materials introduced here were able to replicate the outcomes of the instructors who developed the materials in terms of student learning gains and student feedback.

We introduce participants to Peer Instruction (PI): an active learning technique applicable to the teaching of many subjects, including CS. In PI, Students work together to exchange perspectives and answer challenging conceptual questions, and are supported by short teaching segments. We will introduce and motivate PI, demonstrate its use in combination with a clicker system, and show that PI is much more than the use of clickers. Participants will work in groups to develop new PI questions addressing challenges to their students' learning, and discuss numerous pedagogical benefits conferred through PI.

Peer Instruction (PI) is an instructional approach that engages students in constructing their own understanding of concepts. Students individually respond to a question, discuss with peers, and respond to the same question again. In general, the peer discussion portion of PI leads to an increase in the number of students answering a question correctly. But are these students really learning, or are they just "copying" the right answer from someone in their group? In an article in the journal Science, Smith et al. affirm that genetics students individually learn from discussion: having discussed a first question with their peers, students are better able to correctly, individually answer a second, conceptually-related question. We replicate their study, finding that students in upper-division computing courses (architecture and theory of computation) also learn from peer discussions, and explore differences between our results and those of Smith et al. Our work reveals that using raw percentage gains between paired questions may not fully illuminate the value of peer discussion. We define a new metric, Weighted Learning Gain, which better reflects the learning value of discussion. By applying this metric to both genetics and computing courses, we consistently find that 85-89% of "potential learners" benefit from peer discussion.

Peer Instruction (PI) has a significant following in physics, biology, and chemistry education. Although many CS educators are aware of PI as a pedagogy, the adoption rate in CS is low. This paper reports on four instructors with varying motivations and course contexts and the value they found in adopting PI. Although there are many documented benefits of PI for students (e.g. increased learning), here we describe the experience of the instructor by looking in detail at one particular question they posed in class. Through discussion of the instructors' experiences in their classrooms, we support educators in consideration of whether they would like to have similar classroom experiences. Our primary findings show instructors appreciate that PI assists students in addressing course concepts at a deep level, assists instructors in dynamically adapting their class to address student misunderstandings and, overall, that PI encourages students to be engaged in conversations which help build technical communication skills. We propose that using PI to engage students in these activities can effectively support training in analysis and teamwork skills.

Utility functions can be used to represent the value users attach to job completion as a function of turnaround time. Most previous scheduling research used simple synthetic representations of utility, with the simplicity being due to the fact that real user preferences are difficult to obtain, and perhaps concern that arbitrarily complex utility functions could in turn make the scheduling problem intractable. In this work, we advocate a flexible representation of utility functions that can indeed be arbitrarily complex. We show that a genetic algorithm heuristic can improve global utility by analyzing these functions, and does so tractably. Since our previous work showed that users indeed have and can articulate complicated utility functions, the result here is relevant. We then provide a means to augment existing workload traces with realistic utility functions for the purpose of enabling realistic scheduling simulations.