A two-stage exam (TSE) is an exam format where students write an exam individually and rewrite the same exam in collaboration with a group of peers. This Working Group investigates TSEs in post-secondary computer science courses. By collecting data from a variety of post-secondary institutions, we aim to identify correlations between course environment indicated through the CALI inventory and students' perceptions and performance on TSEs, with consideration for students in under-represented groups. Students will be given surveys upon completion of TSEs which include open-ended questions regarding their group's dynamics and perceptions of participating in a TSE. Instructors will also provide their experiences running and observing TSEs. This research aims to understand best practices for the implementation of TSEs.

This research investigates the effectiveness of randomized computer-based exams used in Carnegie Mellon's Introduction to Computer Systems course. Students received exams with 7-8 multiple choice and short answer questions that were automatically graded with regular expressions. Each question is randomly drawn from a pool of questions within its respective category. The exam system collects data about how each student progresses throughout their exam. This data [1] includes the order in which students view and answer the exam questions, the time spent viewing each question, and the score they have achieved at any given point during their exam. Our analysis of student scores quantifies and validates the fairness of these exams by reproducing prior methodology [2] that uses the MIRT multi-dimensional Generalized Partial Credit Model. The model allows us to estimate question difficulty, student ability, and question discrimination for any given student and exam question. Further analysis examines the correlation of higher-level student behaviors (e.g. order of solving questions) with ability and score. Key features include which question categories students spent the longest on, which ones students scored the highest on, and which ones the majority of students started the exam with. These insights were derived with statistical methods as well as supervised and unsupervised machine learning methods. This research supports the value of administering computer-based exams and informs future exam design. This research also motivates future interventions to test for identifiable behavioral features and students' test-taking strategies.

This work explores the exam data for a second-year course, Introduction to Computer Systems. Specifically, we are analyzing and releasing anonymized versions of the autosave logs for the exams. We can see what question a student was working on at any time, and using the autograding capabilities of the server, what their score is when they switch to a different question.

Most personal student interactions with instructional staff come through office hours. Particularly in large courses, office hours are predominantly run by teaching assistants (TAs). TAs are best advantaged by support and training from more senior instructional staff, especially faculty. This Birds-of-a-Feather session will provide a forum for discussing challenges and innovations in managing office hours towards improving the student learning experience and environment by discussing ideas around (1) personalized support and mentoring for TAs, (2) technological support such as online queues and internal wikis, (3) considerations for remote teaching and learning, and (4) methods for evaluating office hours. It is hoped that these conversations may inspire transformative changes to office hours structure that lead to future educational innovations in research and in practice. While this session will emphasize TA-supported office hours, the discussion may also inspire new ideas for managing instructor office hours as well. During our session we will create breakout rooms for each topic and use a shared document where each room can take notes. After the break out session, each group will report back and create a summary of their discussion notes, which will be publicly archived at https://kevinl.info/office-hours

Most personal student interactions with instructional staff come through office hours. Particularly in large courses, office hours are predominantly run by teaching assistants (TAs). TAs are best advantaged by support and training from more senior instructional staff, especially faculty. This Birds-of-a-Feather session will provide a forum for discussing challenges and innovations in managing office hours, particularly in computer science courses, with the aim of improving the student learning experience and environment. This session will have the opportunity for discussing the mentoring and personalized support of TAs, as well as what technological support exists, such as queue software or internal wikis, and additionally ideas for more significant changes to office hours structure that could lead to future collaborations or research. While this session will emphasize TA-supported office hours, the discussion should inspire new ideas and techniques for managing individual-instructor office hours as well.

Most student personal interactions with the course staff come through office hours. Particularly in large courses, the office hours are predominantly run by teaching assistants (TAs). TAs are best advantaged by support and training from more senior instructional staff, especially faculty. This BOF will provide a forum for discussing mentoring techniques and other individualized support of the TAs (particularly in computer science courses) to improve student learning and experience, as well as longer-term gains to the discipline. Second, we will be discussing what technological support exists, such as queue software or internal wikis, to improve student learning and experience in office hours. Finally, we will discuss ideas for more significant changes in office hour structures that could lead to future collaborations or research experiments.

This work describes our experience in revising one of the major programming assignments for the second-year course Introduction to Computer Systems, in which students implement a version of the malloc memory allocator. The revisions involved fully supporting a 64-bit address space, promoting a more modern programming style, and creating a set of benchmarks and grading standards that provide an appropriate level of challenge. With this revised assignment, students were able to implement more sophisticated allocators than they had in the past, and they also achieved higher performance on the related questions on the final exam.

Research shows the benefit of using active learning in computer science education; however, only limited resources (such as, prior publications) exist for systems courses (including architecture, networking, operating systems). This BoF brings together practitioners of various levels of experience to discuss ways to augment or replace traditional lecturing. We will discuss different techniques, possible materials available, and results measured. This BoF should benefit both instructors considering adopting techniques and instructors looking to discuss issues with their usage.

Active learning techniques are increasingly used in lower-level Computer Science courses. This work explores the use of active learning techniques in a graduate Computer Science course on computer architecture, where the course enrollment is composed of both undergraduates and graduate students. Initial results are presented on how the two groups of students respond differently to the techniques. In particular, the study includes the effect of using POGIL in place of a subset of lectures, measured through both student responses and test scores.

Parallel programs can be characterized by task graphs encoding instructions, memory accesses, and the parallel work’s dependencies, while representing any threading library and architecture. This article presents Contech, a high performance framework for generating dynamic task graphs from arbitrary parallel programs, and a novel representation enabling programmers and compiler optimizations to understand and exploit program aspects. The Contech framework supports a variety of languages (including C, C++, and Fortran), parallelization libraries, and ISAs (including × 86 and ARM). Running natively for collection speed and minimizing program perturbation, the instrumentation shows 4 × improvement over a Pin-based implementation on PARSEC and NAS benchmarks.

Many new non-volatile memory technologies have been considered as a future scalable alternative to DRAM. Memory technologies such as MRAM, FeRAM, PCM have emerged as the most viable alternatives. But these memories have limited wear endurance. Practically realizable main memory systems employing these memory technologies are possible only if the wear across these memories is reduced as well as uniformly distributed. Limited endurance has resulted in extensive wear leveling research with the goal of uniformly distributing write traffic throughout available physical memory. Basic support for wear leveling is already present in existing systems, in the form of operating system paging. The Operating System (OS) changes virtual to physical translations over time. As a result, write traffic is naturally spread out. Proper evaluation of the need for wear leveling as well as the impact of the corresponding technique must take this phenomenon into account. Ignoring the effect of OS paging mechanism can result in highly inaccurate memory lifetime extrapolations. We demonstrate through simulation results, the effects of inaccurate extrapolations in the absence of OS modeling. Accurate memory lifetime simulation can take from many months to years. Although sampling techniques are commonly employed for speedup, our results show that naïve extrapolation techniques can lead to wildly different lifetime estimates. We show how sampling can be accurately applied by accounting for the different components in the write stream observed by main memory. Finally, we present a heuristic to quickly estimate memory lifetime for a given application.

Data structure selection is one of the most critical aspects of developing effective applications. By analyzing data structures' behavior and their interaction with the rest of the application on the underlying architecture, tools can make suggestions for alternative data structures better suited for the program input on which the application runs. Consequently, developers can optimize their data structure usage to make the application conscious of an underlying architecture and a particular program input.