WIP Research Paper: Identifying the Threshold Concepts within a discipline illuminates the key concepts or components within the curriculum. Once students have overcome the barrier of learning these concepts, they often will be better able to identify as members of that community, and understanding a threshold concept opens the door to learning additional concepts. Within computer science, there has been much debate over what concepts could potentially be threshold concepts. Meyer and Land originally defined threshold concepts as resulting in an individual being placed into a state of uncertainty or liminality, and successfully traversing this liminal state results in a transformation of the individual with potential feelings of accomplishment. While there has been some work attempting to identify threshold concepts within the first year or beginning stages of programming, little work has considered the intermediate years (years 2 and 3) of university study and what potential threshold concepts exist during this time period. Our goal with this work is to help address this gap that exists by answering the following research question: What do intermediate students identify as being troublesome and/or ‘uncomfortable to learn’ within their computer science coursework?” A first cohort of participants were interviewed in late 2022 and coding began in the first quarter of 2023. The coding of these interviews proved challenging. The students who were interviewed often did not give enough information about a concept for the coders to identify whether the concepts had the key characteristics of threshold concepts. It was considered and accepted that the original interview protocol was not supporting the participants well in eliciting the types of information needed to identify a concept as threshold. The interview protocol was redesigned, and new interviews commenced. The work presented here is a continued discussion of the initial findings and the subsequent change in interview protocol, with the primary improvement being an inclusion of concept mapping. Concept maps, or a graphical representation of the interrelationship of topics and ideas, coupled with an intentional simplification of associated terminology, are expected to reduce cognitive load as participants reflect on their learning experiences. Interviews with the revised protocol including concept maps have been more engaging and productive in identifying potential threshold concepts within the intermediate computer science curricula.

It is often the case that computer science classrooms use traditional grading practices where points are allocated to assignments, mistakes result in point deductions, and assignment scores are combined using some form of weighted averaging to determine grades. Unfortunately, traditional grading practices have been shown to reduce achievement, discourage students, and suppress effort to such an extent that some common elements of traditional grading practices have been termed toxic. Using grades to reward or punish student behavior does not encourage learning and instead increases anxiety and stress. These toxic elements are present throughout computing education and computer science classrooms in the form of late penalties, lack of credit for code that doesn't compile or pass certain unit tests, among others. These types of metrics, that evaluate behavior are often influenced by implicit bias, factors outside of the classrooms (e.g., part-time employment), and family life situations (e.g., students who are caregivers). Often, students in these situations are disproportionately from low-socioeconomic backgrounds and predominantly students of color. Through this paper, we will present a case for adoption of equitable grading practices and a call for additional support in classroom and teaching technologies as well as support from administrations both at the department and university level. By adopting a community of practice approach, we argue that we can support new faculty making these changes, which would be more equitable and inclusive. Further, these practices have been shown to better support student learning and can help increase student learning gains and retention.

WIP Research Paper: The ability to identify threshold concepts within a discipline recognizes key components within a curriculum which, when learned, enables an individual to demonstrate as a member of that community. Within computer science, potential threshold concepts have sparked debates among researchers. According to the original definition by Meyer and Land, threshold concepts result in an individual being placed into a state of uncertainty or liminality and successful traversal of this liminal state results in a potentially irreversible transformation. When seeking to identify threshold concept, researchers often search for what concepts students feel are ‘troublesome’ or ‘difficult to learn’ since this state is often difficult to describe or understand when currently inside it or just past it. While threshold concepts have been identified for the beginning stages of programming, there is little to no work on the intermediate years of university computing education (years 2 & 3) and what potential threshold concepts exist during that time for computer science students. Our goal with this research is to help address this gap by answering the following research question: What concepts do intermediate students identify as being troublesome and/or ‘uncomfortable to learn’ within their computer science coursework?

In our CS1 course, we found that students were failing at a rate similar to that reported in literature. In an effort to improve this DFW rate, we made a few changes to the course. While the changes led to some improvement, the results were not as significant as we expected. We found that in our CS1 course, first-semester students were passing at a significantly higher rate than non-first-semester students. Upon looking at other STEM and non-STEM courses at the university, we noted that the institution's first term calculus and introductory English courses had similar patterns, however the introductory sociology course did not. Further data analyses challenged several other of our beliefs about who was failing the course. Our findings leave us with several questions. Is this pattern specific to our university? Is this a particular course problem? From the perspective of the discipline, is there something in this data that points to a set of problems for computing education researchers to consider? This poster serves as both a presentation of our initial experiential results and an attempt to start a conversation amongst community members around whom actually is struggling within our early CS courses.

This lightning talk describes our current efforts to isolate and identify potential threshold concepts and threshold capabilities which are experienced by intermediate computer science undergraduate students. We are utilizing the theoretical framework as described by Meyer, Land, Bowden, and Baillie. Two of the key identifiable characteristics surrounding a threshold concept are (1) the transformation one undergoes as a result of incorporating the concept into their knowledge and (2) the state of uncertainty (liminality) experienced as the learner develops this understanding. As a result of this transformation and the resultant traversal of a liminal state, elements of an identity are often developed. For the computer science undergraduate, we hypothesize that advanced computational thinking skills are developed along with these transformations leading to the development of a computer scientist identity. Through identification of the threshold concepts/capabilities experienced by intermediate students the potential corollary concepts behind computational thinking may be uncovered as well. The overall project intends to integrate elements of threshold concepts, liminality, and transformative learning to produce a framework through which computational thinking can be probed and students' progression can be better assessed and supported. These efforts are only beginning - curious faculty and students are invited to attend our presentation and consider potential collaborative participation in our endeavors.

Each year, we graduate a fair number of students with college degrees in computing-related disciplines. Each year, we lose a fair number of students out of the discipline. We spend a good deal of time focusing on what is causing students to leave or fail. We do not spend a lot of time focusing on the learning journey and supporting students throughout their transformative experiences. Threshold concepts are those concepts or learning experiences which define the overall disciplinary learning journey, enabling the learner to see from a new perspective, participate in discourse previously unavailable to them, and engage with the world from a transformed frame of understanding. They are often those concepts difficult for students to understand within a discipline such as "personhood' in Philosophy, 'gravity' in Physics, or 'limit' in Mathematics. As a result of this transformation and the resultant traversal of a liminal state, elements of an identity developed. In this poster, we present our current work identifying potential threshold concepts experienced by intermediate computer science students and consider the impact of these concepts on their development of an identity within computer science.

One of the most ubiquitous techniques for evaluating research, particularly in educational settings, has been Likert-style questionnaires. Having a participant rank their level of engagement, agreement, or interest on a scale can provide powerful insight into an individual's perspective and attitude. However, as computing education matures as a discipline it becomes important for us to examine our practices and ensure that we are employing the techniques of evaluation properly. Likert-style questionnaires can be prone to unintentional biases and noise which, from the perspective of the researcher, may affect the study in unknown, unexpected, and potentially undesirable ways. In this research we seek to aid the computing education researcher not only avoid biases and noise but also improve the reproducibility of their work. First, we establish best practices for these instruments by synthesizing recommendations from the original creator, Rensis Likert, the Center for Disease Control (CDC), the Association of American Medical Colleges (AAMC), and the American Association for Public Opinion Research (AAPOR). We then considered additional sources of unintentional biasing and noise resulting from the measurement scales/response options, specifically possible biasing resulting from how response options are presented to the study participant. With these recommendations, we then examined 121 evaluation instruments used in computer science education research and curated in the csedresearch.org online database to see how often researchers unintentionally fall victim to the pitfalls of ambiguity, awkward phrasing, use of conjunctions, leading or biased statements, and double negatives. We found that the occurrence of at least one of these problematic statements to be in 82.6% of all instruments. We also examine demographic information for the intended study participants, number of response options, and how those options are presented. Overall, while we see many instrument authors falling victim to a few of these common pitfalls, we believe that increased awareness of potentially problematic statements/measurement scales and their impact on research bias and reproducibility will help insulate computing education researchers from avoidable complications and strengthen the discipline throughout.