Refinement types, a type-based generalization of Floyd-Hoare logics, are an expressive and modular means of statically ensuring a wide variety of correctness, safety, and security properties of software. However, their expressiveness and modularity means that to use them, a developer must laboriously annotate all the functions in their code with potentially complex type specifications that specify the contract for that function. We present LHC, a neurosymbolic agent that uses LLMs to automatically generate refinement type annotations for all the functions in an entire package or module, using the refinement type checker LiquidHaskell as an oracle to verify the correctness of the generated specifications. We curate a dataset of three Haskell packages where refinement types are used to enforce a variety of correctness properties from data structure invariants to low-level memory safety and use this dataset to evaluate LHC. Previously these packages required expert users several days to weeks to annotate with refinement types. Our evaluation shows that even when using relatively smaller models like the 3 billion parameter StarCoder LLM, by using fine-tuning and carefully chosen contexts, our neurosymbolic agent generates refinement types for up to 94% of the functions across entire libraries automatically in just a few hours, thereby showing that LLMs can drastically shrink the human effort needed to use formal verification.

While the computer science community has explored the importance of Undergraduate Research Experiences (UREs) and, separately, collaboration in computing (e.g. pair programming), little research has studied collaboration in the context of a URE. We performed a qualitative thematic analysis of how students collaborate within a group-structured, academic-year, inclusive computing URE catered towards second-year students at two large public research universities in the United States. We analyzed free-response and Likert-scale survey data collected early and late in the program from a total of 106 students who comprised three program cohorts. We studied their overall group function, what aspects of group work led to positive or negative group experiences, how their group affected their feelings of being supported, and how their group affected their sense of belonging in computing. We found that group experiences were overwhelmingly positive. Further, we found that students’ experiences in groups centered around three themes: group fit and belonging, emotional and academic support, and logistics. Within each theme, their experiences were rich and nuanced, and we observed variations by gender, and to a lesser degree by race. Our work suggests that group-structured UREs are both feasible and beneficial for students, and we give concrete suggestions for how to make these experiences successful.