Abstract We have completed machine-assisted proofs of two highly-optimized cryptographic primitives, AES-256-GCM and SHA-384. We have verified that the implementations of these primitives, written in a mix of C and x86 assembly, are memory safe and functionally correct, by which we mean input-output equivalent to their algorithmic specifications. Our proofs were completed using SAW, a bounded cryptographic verification tool which we have extended to handle embedded x86. The code we have verified comes from AWS LibCrypto. This code is identical to BoringSSL and very similar to OpenSSL, from which it ultimately derives. We believe we are the first to formally verify these implementations, which protect the security of nearly everybody on the internet.

A flexible late submission policy can be used to help address breaches of academic integrity and student stress as they relate to programming assignments. To this end, we examine the use of late days -- earned, penalty free deadline extensions -- during our large format, two-semester introductory programming sequence. In particular, we examine correlations between patterns of student assignment submission and student plagiarism. Lexical-analysis-based techniques are used to identify probable cases of plagiarism across student submissions. Furthermore, we present metrics for estimating student stress at submission time. Our metric utilizes the automated grading score of the student"s current and previous submission attempts, the imminence of the assignment"s deadline, and the availability of late days. We corroborate these patterns with self-reported student surveys conducted at the end of the academic year.

Submitty is an open source programming assignment submission system from the Rensselaer Center for Open Source Software (RCOS) at Rensselaer Polytechnic Institute (RPI) accessed via an online interface. Submitty allows students to submit their code through file upload or version control, such as an internal Git/SVN server or Github, where it is then tested with a highly configurable and customizable automated grader. For each assignment, instructors can specify whether or not students can work in teams. For team assignments, the instructor can either assign teammates or allow the students to choose. In addition to the auto-grading for submissions, Submitty supports human grading. The human graded rubric is developed by the graders as they work, allowing reuse of common feedback messages and partial credit points. The rubric can be searched and modified during and after grading is complete for consistency. By default, grading is handled by instructors and TAs who are assigned to sections of students, which can be rotated through the semester. However, an instructor can choose to incorporate peer grading, which will allow students to anonymously view and submit grades for each other, receiving multiple peer grades per assignment. Submitty has been used at RPI for several years for a variety of courses, serving over 1500 students and 50 instructors and TAs each semester, and has recently been used by several other universities. We will present "case studies" of assignment configurations for autograding and manual grading and demonstrate the grading interface in support of team submissions and peer grading.

With surging enrollment in Computer Science courses at both the introductory and advanced level, it is critical to leverage automated testing and grading to ensure consistent assessment of student learning. Program analysis tools allow us to streamline the grading process so instructors and TAs can spend more time teaching, one-on-one tutoring, and mentoring students. We present complex use cases of automated assignment testing and grading within the open-source homework submission system, Submitty. Students receive immediate and detailed feedback from the automated grader, and can resubmit to correct errors. Submitty uses custom-built grading tools, including difference checking of plaintext program output, instructor authored assignment-specific custom graders, and static analysis tools that reason about program structure. In addition, it employs a variety of external tools, including version control (Git and SVN), unit testing frameworks (JUnit), memory debugging tools (Valgrind and DrMemory), and code coverage tools (Emma). In this poster we describe our experience with memory debugging and code coverage tools, and outline plans to include immutability inference and verification.

Student experience in introductory computer science classes can be enhanced by applying static analysis techniques to automatically grade assignments. At Rensselaer Polytechnic Institute (RPI), introductory computer science classes (using Python) exceed 650 students in size. As resources are limited, it is infeasible to have teaching staff individually examine each student's answer for small in-lecture exercises; however, qualitative data regarding student code independent from execution is still valuable (and in some cases required) to assess progress. When static analysis utilities were made available to instructors and integrated with automatic assignment testing, instructors were able to judge student performance and provide feedback at a scale that would otherwise be infeasible.

Submitty (http://submitty.org) is an open source programming assignment submission system from the Rensselaer Center for Open Source Software (RCOS) at Rensselaer Polytechnic Institute (RPI). Students can submit their code via a web interface in a variety of ways, where it is then tested with a highly configurable and customizable automated grader. Students receive immediate feedback from the grader, and can resubmit to correct errors as needed. Through an online interface, TAs can access detailed grading results and supplement the automated scores with manual grading (numeric and written feedback) of overall program structure, good use of comments, reasonable error checking, etc. and any non-programming components of the assignment. The instructor can also configure the system to allow for a configurable late day policy on a per assignment and per student basis. We currently use Submitty in eight different courses (spanning from introductory through advanced topics) serving over 1500 students and 35+ instructors and TAs each week. We will present a range of "case study" assignment configurations in a hands-on demo, going from simple through complex, using a variety of different automated grading methods including per-character and per-line output difference checkers, external unit testing frameworks (such as JUnit), memory debugging tools (Valgrind and DrMemory), code coverage (e.g., Emma), static analysis tools, and custom graders. Submitty can be customized per test case as appropriate to apply resource limits (running time, number of processes, output file size, etc.) and to display or hide from students the program output, autograding results, and testing logs.

The Rensselaer Polytechnic Institute (RPI) Homework Submission Server is an ongoing open source project used to collect, compile, and automatically grade programming homeworks for students in our introductory and sophomore computer science classes. It allows viewing of homework, lab, test, and overall grades and late submissions and excused absences on homework. Our first hypothesis is that an electronic submission server is the preferred way for students to submit their coding homeworks because it provides immediate feedback about the correctness of their code and ensures fair, consistent grading since their code is compiled and run with the same test cases, on the same computer. Our second hypothesis is that students appreciate courses with a flexible policy for late submission of homeworks, allowing them to use a specific number of "late days" throughout the semester without penalty. We recently conducted a survey to test these hypotheses and will incorporate specific feedback from the students as we continue development and expand the server to more courses at RPI and other universities.