In many CS educational research studies, students are surveyed to understand their reactions to a particular pedagogical approach or tool. These surveys, as well as other types of evaluations, often invite students to provide open-ended feedback about their experiences. However, analyzing these comments can prove to be a challenge, especially to CS educators who may not have strong expertise in qualitative research methods. In addition, in a large study, evaluating all of the provided comments can consume a significant amount of researcher time. In this work, we undertook two separate conversations with ChatGPT in which we prompted it to perform qualitative analysis of a set of comments collected in an earlier study. This allowed us to begin to judge how effectively a modern large language model can serve as an assistant in qualitative analysis. We found that with the prompts we used, ChatGPT can reliably build a set of reasonable labels (codes) for a set of comments, but the application of its labels to specific comments may or may not be effective and human researchers still need to use care and their own understanding in interpreting its output.

Context-based assignments have been shown as effective and popular for introductory-level computing courses. We study the use of one such context, pencil puzzles (puzzles typically found in newspapers), in upper-level core computing courses. These puzzles are designed to inspire computational thinking, making them a great choice for introductory-level computing assignments, but their fit for upper-level courses is less clear. We collaborated with several instructors of upper-level courses at four institutions, who delivered a pencil-puzzle-based assignment in their course and allowed us to survey their students about their experience. Overall, the students indicated positive perceptions of the assignments. The most varied answers related to implementation aspects of the assignments. To analyze correlations between students' sentiments and their demographic and experiential background, we used mixed-effects regression modeling to analyze this heterogeneous data set. The survey responses were characterized by two dimensions, one roughly corresponding to students' sentiment about the assignment and the other to their technical assessment of the assignment. For the first dimension, we found that the students' self-reported level of preparedness from earlier courses positively correlated with their enjoyment of and satisfaction with the pencil puzzle assignment. The second dimension was correlated with both the level of preparedness as well as the students' self-reported problem solving type: Clarifier, Implementor, Ideator, and Developer. Somewhat surprisingly, the analysis indicated Ideator as being the most positively correlated with the technical aspects of the assignment. Notably, the analysis did not indicate any correlation with students' race or gender in either dimension.

We report on our experience using common natural language processing (NLP) tools to analyze two vastly different data sets of free-form responses collected during a study of assignments in introductory computing courses. Our first data set consists of typically short comments left by hundreds of students on assignment surveys. Our second data set is comprised of semi-structured individual interviews of eight instructors of up to an hour long each. We collected the data across several years as part of our investigation of the use of pencil puzzles as a context for introductory computer science. In an earlier work, we manually analyzed a fraction of the student comments (all data collected until that point), using grounded theory. The results were illuminating, but the process was very time consuming, consisting of manual assignment of a small number of codes to each comment. In this work, we investigate the usability of common NLP tools to speed up the process for the entire data set of student comments. We also applied these tools to the instructor interviews. The NLP tools do not appear to be effective to create the code base, but, once the code base was determined, they performed the actual coding (assignment of codes to each student comment) promisingly well. For the long-form instructor interviews, the situation was much more challenging, due to the wide-ranging nature of semi-structured interviews, interleaving discussion topics, and elements of natural speech. We report on the lessons learned while automatically analyzing these complex data sets.

Complex systems across various domains can be naturally modeled as signed networks with positive and negative edges. In this work, we design a new class of signage models and show how to select the model parameters that best fit real-world datasets using maximum likelihood.

Signed networks (networks with positive and negative edges) commonly arise in various domains from molecular biology to social media. The edge signs -- i.e., the graph signage -- represent the interaction pattern between the vertices and can provide insights into the underlying system formation process. Generative models considering signage formation are essential for testing hypotheses about the emergence of interactions and for creating synthetic datasets for algorithm benchmarking (especially in areas where obtaining real-world datasets is difficult). In this work, we pose a novel Maximum-Likelihood-based optimization problem for modeling signages given their topology and showcase it in the context of gene regulation. Regulatory interactions of genes play a key role in the process of organism development, and when broken can lead to serious organism abnormalities and diseases. Our contributions are threefold: First, we design a new class of signage models for a given topology, and, based on the parameter setting, we discuss its biological interpretations for gene regulatory networks (GRNs). Second, we design algorithms computing the Maximum Likelihood -- depending on the parameter setting, our algorithms range from closed-form expressions to MCMC sampling. Third, we evaluated the results of our algorithms on synthetic datasets and real-world large GRNs. Our work can lead to the prediction of unknown gene regulations, novel biological hypotheses, and realistic benchmark datasets in the realm of gene regulation.

The use of a well-chosen context for course assignments is widely regarded as motivating for students. However, it is challenging to study the utility of bringing a particular context to computing courses across different types of institutions and student demo- graphics. This is especially true in introductory computing since courses vary widely, for example, in topic order and depth of coverage. In this experience report, we present our approach to, and lessons learned from, studying the efficacy of a specific context for introductory computing assignments across a variety of environments. We focus on the context of pencil puzzles (puzzles like Sudoku or crosswords, designed to be solved on paper using a pencil) and the deployment and fit of pencil-puzzle-based assignments across different institutions' introductory curricula. We describe our overall process, including recruitment of instructors from a variety of institutions, development and deployment of assignments, and collection of student grade and survey data (including all necessary approvals). By design, we did not use the same assignment at each university, since we aimed to study the underlying context rather than a specific assignment, while also establishing the adoptability of the context to different circumstances. We discuss the heterogeneity of the resulting data set, how we chose to analyze it, and what conclusions can (and cannot) be drawn from such data. We conclude with lessons learned from this experience, with the hopes that they can help others who wish to propagate their innovations and study them in diverse situations.

Assignments based on meaningful real-world contexts have been shown to be valuable in introductory computing education. However, it can be difficult to distinguish the value of a broad context from the value of a particular instantiation of that context. In this work in progress, we report on our initial findings gathered from deployments of different pencil-puzzle-based assignments. Specifically, we have investigated the use of pencil puzzles as a contextual domain, working with instructors at eight institutions to deliver assignments appropriate to their situation and aligning with their existing materials. We then evaluate the assignments using student grades and survey responses regarding student perceptions of the assignments including self-assessed learning, given a wide array of demographic variables. Our initial results show that while there was some dependency of student responses on their prior programming experience, and female students' feedback were more positive about one aspect, overall these types of assignments do not appear to put particular groups of students at a strong (dis)advantage.

Stable matching models are widely used in market design, school admission, and donor organ exchange. The classic Deferred Acceptance (DA) algorithm guarantees a stable matching that is optimal for one side (say men) and pessimal for the other (say women). A sex-equal stable matching aims at providing a fair solution to this problem. We demonstrate that under a class of correlated preferences, the DA algorithm either returns a sex-equal solution or has a very low sex-equality cost.