Large introductory CS courses serve students with heterogeneous programming backgrounds; those with informal preparation fall into a gray area that existing placement mechanisms fail to serve. We present a framework for designing scalable, psychometrically validated placement exams anchored to CS0 final performance and using Item Response Theory (IRT) to validate against institutional benchmarks. Applying this framework at our institution with CS0 completers (N = 236) as the benchmark, we found that students placed into CS1 had ability levels that matched the top 15% of CS0 completers, with 80% achieving A or B in CS1, and showed exceptional separation in ability from CS0-placed students (Hedges' g = 2.84). We make three key contributions: (1) A data-driven psychometric framework enabling iterative optimization of exam items and placement criteria against local benchmarks; (2) Demonstration that PrairieLearn's algorithmic randomization provides exam security while enabling rapid scaling from 19-student pilot to 236-student assessment; (3) Post-hoc analysis suggesting ability estimates (?) enable more precise placement than raw scores, with item difficulties remaining stable across cohorts. This reproducible framework enables institutions to develop locally anchored placement exams on auto-grading platforms that support randomized question generation.

Large CS courses struggle to scale grading, deliver concept-specific practice, and maintain alignment between lecture content and assignments. We present Caliber, a fully functional prototype of an AI-assisted mastery-based platform currently being piloted with a cohort of undergraduate students, designed to help instructors manage exercises, support grading workflows, and deliver targeted practice aligned to demonstrated student needs while preserving instructor control over curriculum and pedagogy. The platform also serves as a research infrastructure for studying AI-assisted grading, mastery progression, and course-content alignment in CS education.

This paper reports on the longitudinal impacts of an inclusive, structured research experience program for early career undergraduates in computer science that engages a large number of students from minoritized groups. We compared academic performance and retention in the major for program participants at two large public research universities in the United States vs. a matched control group of demographically and academically similar students. We found that the retention rate of program participants was higher than the control at both universities, though not statistically significantly so. We found no significant difference in post-program GPA, and the program did not erase equity gaps in GPA by race and first generation status that existed before the program. These results help us understand the benefits and limitations of large-scale early research programs for increasing equity in computer science.

Engaging undergraduates in research has been shown to improve retention, increase students' sense of computer science identity, and increase their chances of continuing to graduate school. Yet research experiences at most universities are ad hoc, and many undergraduates-particularly those from groups underrepresented in computing-do not have the opportunity to participate. The Early Research Scholars Program (ERSP) is a structured, academic-year group-based undergraduate research program designed to help universities vastly increase participation in research for early computing undergraduates. ERSP launched at UC San Diego in 2014 where it now annually engages over 50 second-year undergraduates, 59% of whom are women, and 22% of whom are from underrepresented racial and ethnic groups. The program's portable design has enabled its expansion to 7 other colleges and universities. This workshop will train participants in launching ERSP (or any part of it) at their university to increase and diversify the undergraduates participating in research. Workshop leaders are the ERSP directors at four universities. They will address how to launch and run the program in different contexts. They will provide an interactive, hands-on experience of running the program covering the following topics: developing and teaching a research methods class, student application and selection to ensure a diverse and supportive cohort, and creating a dual-mentoring structure to engage and retain early undergraduates without overburdening faculty. Workshop participants will be invited to join the ERSP virtual community to get support launching their own version of ERSP.

The Early Research Scholars Program (ERSP) was launched in 2014 at UC San Diego as a way to provide the benefits of research experiences to a large and diverse group of students early in their undergraduate computing career. ERSP is a structured program in which second-year undergraduate computing majors participate in a group-based, dual-mentored research apprenticeship over a full academic year. In its first four years ERSP engaged 139 students with a high proportion of women (68%) and racially minoritized students (19%), and participation in ERSP correlated with increased class grades. In 2018 we partnered with three additional universities to launch their own version of ERSP. Implementations at our partner sites have seen similar diversity and initial success, and have taught us how to implement the program in different contexts (e.g. quarters vs. semesters, different credit structures). This paper describes the structure of ERSP and how it can be adapted to different contexts to construct a scalable and inclusive research experience for early-career undergraduates in computing and related fields.

Despite the documented importance of mentoring in undergraduate research, few studies examine how students---especially early undergraduates in computing---perceive their relationships with their mentors. We present a qualitative thematic analysis of the mentoring practices used in an inclusive, structured computer science research program targeting second-year undergraduates across two large public research universities in the United States. Uniquely in this program, students had two mentoring sources: a technical mentor for each research group and a graduate student mentor common to all groups. We analyzed reflections on mentoring from 64 undergraduate researchers at two points in the program. We compared the roles of the two mentors, characterized students' perceptions of both successful and unsuccessful mentors, and examined how mentoring relationships evolved. Generally, students valued mentors who provided project guidance or technical support and who were perceived to be friendly. We found that the roles of the two mentors were complementary in sometimes surprising ways. Overall, our analysis confirms prior work on undergraduate research mentoring, and provides new insights into the unique benefits of a dual-mentoring approach and how to best support early undergraduate computing researchers.

Andrew Gaut, Tony Sun, Shirlyn Tang, Yuxin Huang, Jing Qian, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, William Yang Wang. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020.

Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, William Yang Wang. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.

Undergraduates have been an important part of the teaching staff at many universities for decades. Recent work such as the Peer Teaching Summit at SIGCSE 2019 [1] and a systematic literature review [2] have focused more attention on issues related to the use of undergraduates in teaching assistant roles. This BOF provides a forum to discuss open research questions about undergrad TA/mentor programs at various stages in their evolution. Attendees will have an opportunity to discuss research questions, research methods, and explore possible collaborations. Discussion Leader(s): Diba Mirza will open the session by summarizing the key findings of literature review on UTAs from ICER'19 [1]-in particular, the fact that while there is widespread consensus that using UTAs is a good idea, the evidence backing up this consensus is mostly anecdotal. This creates many opportunities to establish the effectiveness of current practices, and the claimed benefits of the use of UTAs through more rigorous research and to discuss innovative ways to incorporate UTAs in teaching outside of what has been reported in the literature. This will be followed by a themed discussion to brainstorm about research within each of these areas. Based on the large turnout at the BOF on Undergraduate Teaching Assistants in SIGCSE'19, we plan to organize the discussion in smaller groups. Diba Mirza, Phillip Conrad, and Cynthia Lee will lead the discussion within each group. The leads will also document the discussions and share it with the participants.

Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, William Yang Wang. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019.

We present a systematic literature review of the prior work on Undergraduate Teaching Assistants (UTAs) in Computer Science with two goals: (1) to create a taxonomy of practices that relate to the design and implementation of UTA programs, (2) to identify the benefits of using UTAs as claimed by the literature and characterize the level of evidence for those claims. We analyze 336 excerpts from 40 papers related to these goals. We use content analysis on excerpts describing practices to extract high-level themes that include recruiting, UTA and program coordinator duties, training, evaluation and organization of UTA programs. We perform a more fine-grained analysis within each theme to identify specific questions about UTA programs and the answers provided by the literature. Using a similar technique, we report on the claimed benefits of UTA programs to students, UTAs, instructors and institutions. Our analysis follows well-defined protocols involving multiple reviewers and we report on the inter-rater reliability. The results provided in this paper lay the groundwork for developing evidence-based best practices in UTA programs and inform practice and policy related to the use of UTAs at tertiary institutions. As such, it is relevant to educators establishing a new UTA program, expanding an existing program, or continuously improving an established program, as well as those designing research studies of such programs.

Undergraduates have been an important part of the teaching staff at many universities for decades, but this is not a universal practice. This BOF provides a forum to discuss undergraduate TA/mentor programs at various stages in their evolution. Attendees will have an opportunity to discuss the benefits, best practices, and research questions related to the use of undergraduates as teaching assistants and/or mentors. The audience is expected to consist of faculty that have already implemented such programs, and those who might be considering doing so, and want to share information about successes and challenges. Discussion Leader(s): Phill Conrad will facilitate the discussion and invite Colleen Lewis, Cynthia Lee and Diba Mirza to briefly speak about the Undergraduate TA/Mentor programs at their respective institutions. During the balance of the time participants will be invited to contribute ideas or ask questions related to the use of undergraduate TAs and mentors. Expertise of Discussion Leader(s): Diba Mirza and Phill Conrad are faculty in the Computer Science department at UC Santa Barbara, where they have created an undergraduate mentor program in the past two years. Colleen Lewis is an Associate Professor of CS at Harvey Mudd College, and is the lead PI for the csteachingtips.org project, a project to collect and document CS pedagogical content knowledge. Cynthia Lee is a Lecturer in CS at Stanford University, and works closely with the longstanding undergraduate TA program in the department.

Many ocean processes, both biological and physical, greatly depend on and interact with the intrinsic current dynamics of the underwater environment. A promising approach to understand small and large scale spatio-temporal correlations of these processes is to deploy a networked swarm of drifters that float freely with ocean currents. They form a coordinated distributed sampling system that observes ocean processes within their own moving frame of reference. As data interpretation is impossible without knowledge of sampling positions, a method is required to localize the drifters. Furthermore, the localization has to be repeated periodically as the network topology changes due to the inherent motion of the vehicles. This paper proposes a novel energy-aware, distributed solution based on inter-drifter range measurements. It leverages the realization that actual position estimation can be performed after the mission is over. The proposed broadcast-based solution achieves sufficient localization accuracy with an extremely low overhead: around 0.5 transmissions per node per localization.