We present ASM Visualizer, a tool that is designed to help students learn assembly programming, aiding in their understanding of how assembly instructions are executed and the relationship between assembly and equivalent high-level language code.Our tool allows a user to step both forward and backward through the execution of an assembly program, one instruction at a time, seeing how instructions use and modify values in stack memory and CPU registers.ASM Visualizer presents three user-interface modes, supporting different stages of learning assembly programming.Beginners can step through basic arithmetic instructions, whereas more advanced learners can trace through function call/return sequences, stack frame manipulation, or entire assembly programs.We present our experiences using ASM Visualizer in introductorylevel courses at our two institutions, and we discuss other ways in which our tool could be used by educators in both introductory and advanced CS courses.Results from a preliminary assessment of students using our tool show that students gain confidence in their understanding of different aspects of assembly programming.We feel that the visual interface to assembly code execution that ASM Visualizer provides is key to helping students understand assembly.

We present the curricular design, pedagogy, and goals of an introductory-level course on computer systems that introduces parallel and distributed computing (PDC) to students who have only a CS1 background. With the ubiquity of multicore processors, cloud computing, and hardware accelerators, PDC topics have become fundamental knowledge areas in the undergraduate CS curriculum. As a result, it is increasingly important for students to learn a common core of introductory parallel and distributed computing topics and to develop parallel thinking skills early in their CS studies. Our introductory-level course focuses on three main curricular goals: 1) understanding how a computer runs a program, 2) evaluating system costs associated with running a program, and 3) taking advantage of the power of parallel computing. We elaborate on the goals and details of our course's key modules, and we discuss our pedagogical approach that includes active-learning techniques. We also include an evaluation of our course and a discussion of our experiences teaching it since Fall 2012. We find that the PDC foundation gained through early exposure in our course helps students gain confidence in their ability to expand and apply their understanding of PDC concepts throughout their CS education.

A Concept Inventory (CI) is an assessment to measure student conceptual understanding of a particular topic. This article presents the results of a CI for basic data structures (BDSI) that has been previously shown to have strong evidence for validity. The goal of this work is to help researchers or instructors who administer the BDSI in their own courses to better understand their results. In support of this goal, we discuss our findings for each question of the CI using data gathered from 1,963 students across seven institutions.

Computer Science now has a number of validated instruments available for measuring student knowledge or interest in computing including the Second CS1 Assessment (SCS1), The Basic Data Structures Inventory (BDSI), the Computing Attitudes Survey (CAS), and the Digital Logic CI. These instruments can be used by instructors to assess their students and/or their own teaching. They can also be used by researchers to measure students' learning or attitudes. The goal of this BOF is to help instructors and researchers gain a better understanding of how to use these instruments, whether that be to get started in education research, to compare student learning across terms/curricular revisions, or just to learn more about student misconceptions. We will begin by discussing the available instruments, their purpose, and how to obtain them. Then we will open the discussion to the group on what they would like to measure, how these instruments might work for them, and how to best employ them with their students.

The expense of textbooks is a common source of frustration among students. Furthermore, the lack of affordable textbooks can inadvertently limit who studies computer science. One way educators and faculty can mitigate these issues is through adopting (and writing) free online textbooks. This panel discusses the benefits, challenges, and development practices of free online textbooks. The panelists, who have authored three widely used textbooks, characterize the role of free textbooks in CS education, describe their experiences writing a free textbook, and offer advice to faculty interested in incorporating them into their courses.

This paper presents our experiences, motivations, and goals for developing Dive into Systems [17], a new, free, online textbook that introduces computer systems, computer organization, and parallel computing. Our book's topic coverage is designed to give readers a gentle and broad introduction to these important topics. It teaches the fundamentals of computer systems and architecture, introduces skills for writing efficient programs, and provides necessary background to prepare students for advanced study in computer systems topics. Our book assumes only a CS1 background of the reader and is designed to be useful to a range of courses as a primary textbook for courses that introduce computer systems topics or as an auxiliary textbook to provide systems background in other courses. Results of an evaluation from students and faculty at 18 institutions who used a beta release of our book show overwhelmingly strong support for its coverage of computer systems topics, its readability, and its availability. Chapters are reviewed and edited by external volunteers from the CS education community. Their feedback, as well as that of student and faculty users, is continuously incorporated into its online content. We anticipate releasing version 1.0 of the book in spring of 2021, and a release candidate is currently available at https://diveintosystems.org.

Computer Science now has a number of validated instruments available for measuring student knowledge or interest in computing (SCS1, BDSI, CAS, Digital Logic CI, etc.). But when and how should instructors and researchers use these instruments? In this BOF, we will begin by discussing the available instruments, their purpose, and how to obtain them. Then we will open the discussion to the group on what they would like to measure, how these instruments might work for them, and how to best employ them with their students.

Concept inventories (CIs) allow researchers and practitioners to measure student conceptual learning within a course or topic area. While they have enabled meaningful pedagogical change in other disciplines, there are relatively few CIs in computer science. In this paper, we report on our experiences as recent developers of a CI for basic data structures. We discuss each step along the route to a CI and offer tips based on what we have learned. We encourage others to create CIs, and we hope that this paper will serve as a practical guide through the process.

A Concept Inventory (CI) is a validated assessment to measure student conceptual understanding of a particular topic. This work presents a CI for Basic Data Structures (BDSI) and the process by which the CI was designed and validated. We discuss: 1) the collection of faculty opinions from diverse institutions on what belongs on the instrument, 2) a series of interviews with students to identify their conceptions and misconceptions of the content, 3) an iterative design process of developing draft questions, conducting interviews with students to ensure the questions on the instrument are interpreted properly, and collecting faculty feedback on the questions themselves, and 4) a statistical evaluation of final versions of the instrument to ensure its internal validity. We also provide initial results from pilot runs of the CI.

To be effective instructors and CS education researchers, we must identify and understand student difficulties surrounding core computing topics. This study examines student difficulties with the basic data structures commonly found in CS2 courses. Initial exploration of student thinking began with think-aloud interviews with students. These interviews centered on open-ended questions that were iteratively improved upon based on analysis of interview transcripts. The revised open-ended questions were then posed to 249 students during an end-of-term final exam study session. Using the explanations and justifications included by students, responses to the questions were coded and summarized. This work characterizes the difficulties revealed by student responses, and provides details of their prevalence among the examined student population.

Establishing learning goals for a course allows instructors to design course content to address those goals, helps students to focus their learning appropriately, and enables researchers to assess learning of those goals. In this work, we propose six learning goals for a topic prevalent in CS2 courses: Basic Data Structures. These learning goals arise from reviewing several CS2 courses at a variety of institutions, surveying faculty experts who commonly teach CS2, and meeting and working closely with these experts. We outline our process for creating learning goals, identify important topics underlying these goals, and provide examples of how the goals developed on the path to consensus. We also document that the term "CS2" does not have a unified interpretation within the CS education community and describe how this hurdle influenced our decision to focus on Basic Data Structures.

As cloud computing continues to increase in performance, while simultaneously decreasing in cost, it comes as no surprise that traditional computing tasks are frequently being offloaded to cloud services. This paper presents STOIC, a service model for booting and streaming an operating system from public cloud infrastructure. Having booted with STOIC, users can perform regular activities with few noticeable differences between STOIC and more traditional methods of booting an OS. STOIC makes minimal assumptions about the hardware and software capabilities of the booting client and is compatible with many popular cloud storage providers. We show that STOIC's file streaming is responsive and demonstrate several use cases in which streaming an OS is beneficial when compared to alternative file distribution methods.

We present a model for incorporating parallel and distributed computing (PDC) throughout an undergraduate CS curriculum. Our curriculum is designed to introduce students early to parallel and distributed computing topics and to expose students to these topics repeatedly in the context of a wide variety of CS courses. The key to our approach is the development of a required intermediate-level course that serves as an introduction to computer systems and parallel computing. It serves as a requirement for every CS major and minor and is a prerequisite to upper-level courses that expand on parallel and distributed computing topics in different contexts. With the addition of this new course, we are able to easily make room in upper-level courses to add and expand parallel and distributed computing topics. The goal of our curricular design is to ensure that every graduating CS major has exposure to parallel and distributed computing, with both a breadth and depth of coverage. Our curriculum is particularly designed for the constraints of a small liberal arts college, however, much of its ideas and its design are applicable to any undergraduate CS curriculum.

Simulated annealing algorithms, which repeatedly make small changes to candidate solutions to find approximately optimal ones, are a common method for approximating solutions to computationally expensive optimization problems. While using multiple machines to perform such computations in parallel is attractive as a means to reduce the running time, execution in a cluster environment requires substantial software infrastructure to cope with the challenges of a distributed system. In this paper, we introduce MRAnneal, a framework that simplifies the implementation of parallel simulated annealing algorithms. MRAnneal allows users to explicitly trade-off running time and the quality of approximate solutions by supplying only a small number of automatically tuned parameters. Our experimental results demonstrate that implementing applications using MRAnneal is straightforward and that such implementations yield approximate solutions quickly, even for applications without intuitive serial approximation heuristics.

Concept Inventories (CIs) are short, multiple-choice exams that evaluate a student's understanding of core concepts of a particular course. CIs are especially useful for evaluating the effects of pedagogical techniques and interventions on student learning, as their focus on core concepts means any student should be able to answer the questions, and their use as a pre- and post-test allows them to measure student learning gains. However, CI development requires significant overhead: the current state of the art calls for an intensive, six step process. As a result, there are few CIs available for Computer Science, despite their effectiveness.

This paper presents Blender, a framework that enables network operators to improve tenant performance by tailoring the network's behavior to tenant needs. Tenants may upgrade their provisioned portion of the network with specific features, such as multi-path routing, isolation, and failure recovery, without modifying hosted application code. Network operators may differentiate themselves based on upgrades they offer, creating new upgrades via a light-weight programming interface. Blender safely executes multiple tenants' selections simultaneously across a shared network infrastructure. We show that the Blender model can express and extend recently proposed network functionality on existing SDN networks. We use an OpenFlow-based prototype to quantify Blender's performance and potential for deployment at scale.

Concept Inventory (CI) assessments, which target high-level learning goals, have proven highly valuable for higher education research. These assessments have helped to evaluate pedagogical practices among individual instructors, both within and across institutions, and have hence elevated the level of discourse on education within the community. The success of CIs in physics has inspired similar developments in computer science, with a few CIs now developed for computer science courses. However, the development of a CI typically follows a burdensome process, requiring a significant investment to produce a single CI that may be difficult to deploy due to institutional curricular differences. Furthermore, as our field continues to be shaped by technological advances, a path to faster, more modular CI development is critical.

Operating systems courses often present students with multiple approaches to solve a problem, often with differing trade-offs. While students are more than capable of memorizing the details of these competing approaches, they often struggle to recommend a specific approach and analyze its implications. In particular, we find that students exhibit difficultly in interpreting text-based scenario descriptions in a way that allows them to correctly choose between potential solutions when presented with a high-level, conceptual scenario.

Over the last 10-15 years, our industry has developed and deployed many large-scale Internet services, from e-commerce to social networking sites, all facing common challenges in latency, reliability, and scalability. Over time, a relatively small number of architectural patterns have emerged to address these challenges, such as tiering, caching, partitioning, and pre- or post-processing compute intensive tasks. Unfortunately, following these patterns requires developers to have a deep understanding of the trade-offs involved in these patterns as well as an end-to-end understanding of their own system and its expected workloads. The result is that non-expert developers have a hard time applying these patterns in their code, leading to low-performing, highly suboptimal applications.

This work addresses the need for stateful dataflow programs that can rapidly sift through huge, evolving data sets. These data-intensive applications perform complex multi-step computations over successive generations of data inflows, such as weekly web crawls, daily image/video uploads, log files, and growing social networks. While programmers may simply re-run the entire dataflow when new data arrives, this is grossly inefficient, increasing result latency and squandering hardware resources and energy. Alternatively, programmers may use prior results to incrementally incorporate the changes. However, current large-scale data processing tools, such as Map-Reduce or Dryad, limit how programmers incorporate and use state in data-parallel programs. Straightforward approaches to incorporating state can result in custom, fragile code and disappointing performance.