Teachers who are aware of potential student misconceptions teach better than teachers who do not. In this article, we focus on misconceptions in the context of teaching and learning graph algorithms: we seek to discover student misconceptions about Dijkstra’s shortest-path algorithm and related concepts. We observed and interviewed fourteen students who worked on a visual simulation task involving the algorithm; we qualitatively analyzed these data to explore the students’ mistakes and their underlying reasons. We find, among other things, that students conflate concepts such as spanning tree, fringe, and priority queue and that students may neglect the greedy and dynamic-programming aspects of the algorithm; we also identify usability issues in the visualization tool we employed. These findings suggest that teachers and tool designers need to take great care to help students tease apart the key concepts in graph algorithms.

Background and Context: Over the past year, large language models (LLMs) have taken the world by storm. In computing education, like in other walks of life, many opportunities and threats have emerged as a consequence.

One of the most commonly cited theories in computing education research is cognitive load theory (CLT), which explains how learning is affected by the bottleneck of human working memory and how teaching may work around that limitation. The theory has evolved over a number of decades, addressing shortcomings in earlier versions; other issues remain and are being debated by the CLT community. We conduct a systematic mapping review of how CLT has been used across a number of leading computing education research (CER) forums since 2010. We find that the most common reason to cite CLT is to mention it briefly as a design influence; authors predominantly cite old versions of the theory; hypotheses phrased in terms of cognitive load components are rare; and only a small selection of cognitive load measures have been applied, sparsely. Overall, the theory’s evolution and recent themes in CLT appear to have had limited impact on CER so far. We recommend that studies in CER explain which version of the theory they use and why; clearly distinguish between load components (e.g., intrinsic and extraneous load); phrase hypotheses in terms of load components a priori ; look further into validating different measures of cognitive load; accompany cognitive load measures with complementary constructs, such as motivation; and explore themes such as collaborative CLT and individual differences in working-memory capacity.

The modality effect in multimedia learning suggests that pictures are best accompanied by audio explanations rather than text, but this finding has not been replicated in computing education. We investigate which instructional modality works best as an accompaniment for algorithm visualizations. In a randomized controlled trial, learners were split into three conditions who viewed an instructional video on Dijkstra’s algorithm, with diagrams accompanied by audio, text, or both. We find neither a modality effect in favor of the audio condition nor a verbal redundancy effect in favor of using only a single modality rather than both. Taken together with earlier research, our findings suggest that the modality effect is difficult to apply reliably and computing educators should not rush to integrate audio into visualizations in expectation of the effect. We discuss theoretical viewpoints that future research should attend to; these include alternative part-explanations of the modality effect and attention-based models of working memory, among others.

For many decades the abstract has served as a standalone summary of an academic publication, one that succinctly informs readers of what they might expect to find upon reading the paper. While some publication venues require abstracts to conform with a specified structure, many others, including ITiCSE, leave the structure entirely to the paper's authors. In this paper we report on the components identified in the abstracts of ITiCSE's full papers and working group reports. We examine the abstracts of all 1496 of these publications from 25 years of ITiCSE to determine what structural elements they employ. We also construct something of an ethos of computing education by compiling assertions from the introductions of many abstracts. We find, among other things, that very few abstracts include all of the components that are recommended in a structured abstract; that a number of abstracts consist of nothing but background; that nearly half of abstracts do not include any results; and that nearly five percent of abstracts include references, despite often not having an associated reference list. As an example from the ethos, we find that industry wants people with soft skills, and it is important that we teach our students these skills. Our analysis will guide future ITiCSE authors as they consider how to formulate their own abstracts.

Students sometimes produce code that works but that its author does not comprehend. For example, a student may apply a poorly-understood code template, stumble upon a working solution through trial and error, or plagiarize. Similarly, passing an automated functional assessment does not guarantee that the student understands their code. One way to tackle these issues is to probe students' comprehension by asking them questions about their own programs. We propose an approach to automatically generate questions about student-written program code. We moreover propose a use case for such questions in the context of automatic assessment systems: after a student's program passes unit tests, the system poses questions to the student about the code. We suggest that these questions can enhance assessment systems, deepen student learning by acting as self-explanation prompts, and provide a window into students' program comprehension. This discussion paper sets an agenda for future technical development and empirical research on the topic.

We propose a framework for identifying, organizing, and communicating learning objectives that involve program semantics. In this framework, detailed learning objectives are written down as rules of program behavior (RPBs). RPBs are teacher-facing statements that describe what needs to be learned about the behavior of a specific sort of programs. Different programming languages, student cohorts, and contexts call for different RPBs. Instructional designers may define progressions of RPB rulesets for different stages of a programming course or curriculum; we identify evaluation criteria for RPBs and discuss tradeoffs in RPB design. As a proof-of-concept example, we present a progression of rulesets designed for teaching beginners how expressions, variables, and functions work in Python. We submit that the RPB framework is valuable to practitioners and researchers as a tool for design and communication. Within computing education research, the framework can inform, among other things, the ongoing exploration of “notional machines” and the design of assessments and visualizations. The theoretical work that we report here lays a foundation for future empirical research that compares the effectiveness of RPB rulesets as well as different methods for teaching a particular ruleset.

Programming teachers have a strong need for easy-to-use instruments that provide reliable and pedagogically useful insights into student learning. Currently, no validated tools exist for rapidly assessing student understanding of basic programming knowledge. Concept inventories and the SCS1 questionnaire can offer great benefits; this article explores the additional value that may be gained from relatively simple self-evaluation metrics. We apply a lightweight self-evaluation instrument (SEI) in an introductory programming course and compare the results to existing performance measures, such as examination grades and the SCS1. We find that the SEI has a similar correlation with a program-writing examination as the SCS1 does, although both instruments correlate only moderately with the examination and each other. Furthermore, students are much more likely to voluntarily answer the lightweight SEI than SCS1. Overall, our results suggest that both the SEI and other instruments need to be greatly improved and outline future work towards that end.

Instructional designers, examiners, and researchers frequently need to assess the complexity of computer programs in their work. However, there is a dearth of established methodologies for assessing the complexity of a program from a learning point of view. In this article, we explore theories and methods for describing programs in terms of the demands they place on human cognition. More specifically, we draw on Cognitive Load Theory and the Model of Hierarchical Complexity in order to extend Soloway's plan-based analysis of programs and apply it at a fine level of granularity. The resulting framework of Cognitive Complexity of Computer Programs~(CCCP) generates metrics for two aspects of a program: plan depth and maximal plan interactivity. Plan depth reflects the overall complexity of the cognitive schemas that are required for reasoning about the program, and maximal plan interactivity reflects the complexity of interactions between schemas that arise from program composition. Using a number of short programs as case studies, we apply the CCCP to illustrate why one program or construct is more complex than another, to identify dependencies between constructs that a novice programmer needs to learn and to contrast the complexity of different strategies for program composition. Finally, we highlight some areas in computing education and computing education research in which the CCCP could be applied and discuss the upcoming work to validate and refine the CCCP and associated methodology beyond this initial exploration.

The programming education literature includes many observations that pass rates are low in introductory programming courses, but few or no comparisons of student performance across courses. This paper addresses that shortcoming. Having included a small set of identical questions in the final examinations of a number of introductory programming courses, we illustrate the use of these questions to examine the relative performance of the students both across multiple institutions and within some institutions. We also use the questions to quantify the size and overall difficulty of each exam. We find substantial differences across the courses, and venture some possible explanations of the differences. We conclude by explaining the potential benefits to instructors of using the same questions in their own exams.

Ten selected questions have been included in 13 introductory programming exams at seven institutions in five countries. The students' results on these questions, and on the exams as a whole, lead to the development of a benchmark against which the exams in other introductory programming courses can be assessed. We illustrate some potential benefits of comparing exam performance against this benchmark, and show other uses to which it can be put, for example to assess the size and the overall difficulty of an exam. We invite others to apply the benchmark to their own courses and to share the results with us.

Visual algorithm simulation (VAS) is an activity in which students practice their understanding of algorithms: They simulate the steps of an algorithm by manipulating a bespoke visualization within a supporting software system. Multiple instances of a VAS exercise may be automatically generated for different learners or for repetitive practice by a single learner. In this work-in-progress report, we discuss how misconceptions might be automatically detected in students' solutions to VAS exercises and how misconception-aware feedback might be provided in VAS. We identify two strategies for producing multiple instances of VAS exercises and evaluate them against several criteria. The tradeoffs so identified are also pertinent for similar process-simulation practice beyond VAS and computing education.

We investigated students' use of program visualizations (PVs) that were tightly integrated into the electronic book of an introductory course on programming. A quantitative analysis of logs showed that most students, and beginners especially, used the PVs, even where the PV did not directly affect their grade. Students commonly spent more time studying certain steps than others, suggesting they used the PVs attentively. Nevertheless, substantial numbers of students appeared to gloss over some key animation steps, something that future improvements to pedagogy may address. Overall, the results suggest that integrating PVs into an ebook can promote student engagement and has been fairly successful in the studied context. More research is needed to understand the differences between our results and earlier ones, and to assess the generalizability of our findings.

A small number of recent studies have suggested that learning is enhanced when the illustrations in instructional materials are designed to appeal to the learners' emotions through the use of color and the personification of key elements. We sought to replicate this emotional design effect in the context of introductory object-oriented programming (OOP). In this preliminary study, a group of freshmen studied a text on objects which was illustrated using anthropomorphic graphics while a control group had access to abstract graphics. We found no significant difference in the groups' scores on a comprehension post-test, but the experimental group spent substantially less time on the task than the control group. Among those participants who had no prior programming experience, the materials inspired by emotional design were perceived as less intelligible and appealing and led to lower self-reported concentration levels. Although this result does not match the pattern of results from earlier studies, it shows that the choice of illustrations in learning materials matters and calls for more research that addresses the limitations of this preliminary study.

We analyze the Computing Education Research (CER) literature to discover what theories, conceptual models and frameworks recent CER builds on. This gives rise to a broad understanding of the theoretical basis of CER that is useful for researchers working in that area, and has the potential to help CER develop its own identity as an independent field of study.

No abstract available.

This article brings together, summarizes, and comments on several threads of research that have contributed to our understanding of the challenges that novice programmers face when learning about the runtime dynamics of programs and the role of the computer in program execution. More specifically, the review covers the literature on programming misconceptions, the cognitive theory of mental models, constructivist theory of knowledge and learning, phenomenographic research on experiencing programming, and the theory of threshold concepts. These bodies of work are examined in relation to the concept of a “notional machine”—an abstract computer for executing programs of a particular kind. As a whole, the literature points to notional machines as a major challenge in introductory programming education. It is argued that instructors should acknowledge the notional machine as an explicit learning objective and address it in teaching. Teaching within some programming paradigms, such as object-oriented programming, may benefit from using multiple notional machines at different levels of abstraction. Pointers to some promising pedagogical techniques are provided.

This article is a survey of program visualization systems intended for teaching beginners about the runtime behavior of computer programs. Our focus is on generic systems that are capable of illustrating many kinds of programs and behaviors. We inclusively describe such systems from the last three decades and review findings from their empirical evaluations. A comparable review on the topic does not previously exist; ours is intended to serve as a reference for the creators, evaluators, and users of educational program visualization systems. Moreover, we revisit the issue of learner engagement which has been identified as a potentially key factor in the success of educational software visualization and summarize what little is known about engagement in the context of the generic program visualization systems for beginners that we have reviewed; a proposed refinement of the frameworks previously used by computing education researchers to rank types of learner engagement is a side product of this effort. Overall, our review illustrates that program visualization systems for beginners are often short-lived research prototypes that support the user-controlled viewing of program animations; a recent trend is to support more engaging modes of user interaction. The results of evaluations largely support the use of program visualization in introductory programming education, but research to date is insufficient for drawing more nuanced conclusions with respect to learner engagement. On the basis of our review, we identify interesting questions to answer for future research in relation to themes such as engagement, the authenticity of learning tasks, cognitive load, and the integration of program visualization into introductory programming pedagogy.

This paper presents a preliminary analysis of research papers in computing education. While previous analysis has explored what research is being done in computing education, this project explores how that research is being done. We present our classification system, then the results of applying it to the papers from all five years of ICER. We find that this subset of computing education research has more in common with research in information systems than with that in computer science or software engineering; and that the papers published at ICER generally appear to conform to the specified ICER requirements.