The application of mastery learning, where students progress through their learning in a self-paced manner until they have mastered specific concepts, is considered appealing for teaching introductory programming courses. Despite its growing popularity in computing and its extensive use in other disciplines, there is no overview of the design of courses that use mastery learning. In this position paper, we present an overview of five mastery learning models and discuss examples of how these can be applied in practice, both in foundational programming as well as more advanced courses. Our analysis focuses on the student progression through the course, the assessment structure, and the support for self-paced learning, including for struggling students. This work provides a greater understanding of mastery learning and its application in a computing education context.

Much of the recent work investigating large language models and AI Code Generation tools in computing education has focused on assessing their capabilities for solving typical programming problems and for generating resources such as code explanations and exercises. If progress is to be made toward the inevitable lasting pedagogical change, there is a need for research that explores the instructor voice, seeking to understand how instructors with a range of experiences plan to adapt. In this paper, we report the results of an interview study involving 12 instructors from Australia, Finland and New Zealand, in which we investigate educators' current practices, concerns, and planned adaptations relating to these tools. Through this empirical study, our goal is to prompt dialogue between researchers and educators to inform new pedagogical strategies in response to the rapidly evolving landscape of AI code generation tools.

The paper focuses on a new type of interactive learning content for SQL programming - worked examples of SQL code. While worked examples are popular in learning programming, their application for learning SQL is limited. Using a novel tool for presenting interactive worked examples, Database Query Analyzer (DBQA), we performed a large-scale randomized controlled study assessing the value of worked examples as a new type of practice content in a database course. We report the results of the classroom study examining the usage and the impact of DBQA. Among other aspects, we explored the effect of textual step explanations provided by DBQA.

There has been considerable recent interest in a number of questions related to theory within computing education research. In this editorial, we summarize some of this recent interest, discuss the process in which this special issue came to be, and how papers were selected for inclusion. We end with a brief summary of each of the six papers appearing in this special issue, highlighting the relevance of each to this issue's theme.

Many computer applications receive information about external events and produce responses. Such applications include web server software, many services used via Internet browsers, as well as most graphical user interfaces in devices from mobile phones and home appliances to general-purpose computers and industrial applications. The development of such applications is often called event-driven programming (EDP). Despite the continuously increasing importance of event-driven applications in the modern society, only a little published experimental research has been targeted at teaching and learning EDP. This dissertation addresses this research gap. The dissertation has two major parts, of which the first one addresses understanding EDP. As an introduction, it reflects on the difficulty of defining the term event-driven programming and continues by presenting a mapping review regarding published research on teaching and learning EDP. From there, it proceeds to explore students’ perceptions of concepts such as a button, an event, and an event handler. This discussion is based on three studies that involved questionnaires and exercises for students of two online courses. A few questions included reading program code, and in one exercise, the participants were asked to answer using a concept map. The analysis of these answers revealed fundamental misunderstandings regarding EDP and user interfaces. Based on them, the dissertation offers practical suggestions for improving the teaching practice. The second major part of the dissertation addresses teaching EDP from two perspectives. First, it explores contextualization of teaching EDP and focuses on tools and exercises of an approach called media computation. Second, it presents a prototype of a tool that can be used for visualizing high-level concepts in many online learning materials. This tool supports logging users’ actions, plays well together with version control systems, has a relatively low learning curve for teachers of computer science, as well as is open source and free to use. The tool was evaluated in a small-scale pilot study, which demonstrated its suitability for its intended usage environment.

During the past two decades, event-driven programming (EDP) has emerged as a central and almost ubiquitous concept in modern software development: Graphical user interfaces are self-evident in most mobile and web-based applications, as well as in many embedded systems, and they are most often based on reacting to events. To facilitate both teaching practice and research in programming education, this mapping review seeks to give an overview of the related knowledge that is already available in conference papers and journal articles. Starting from early works of the 1990s, we identified 105 papers that address teaching practices, present learning resources, software tools or libraries to support learning, and empirical studies related to EDP. We summarize the publications, their main content, and findings. While most studies focus on bachelor’s level education in universities, there has been substantial work in K-12 level, as well. Few courses address EDP as their main content—rather it is most often integrated with CS1, CS2, or computer graphics courses. The most common programming languages and environments addressed are Java, App Inventor, and Scratch. Moreover, very little of deliberate experimental scientific research has been carried out to explicitly address teaching and learning EDP. Consequently, while so-called experience reports, tool papers, and anecdotal evidence have been published, this theme offers a wide arena for empirical research in the future. At the end of the article, we suggest a number of directions for future research.

Research into the relationship between learning computing and students' attitudes, beliefs, and emotions often builds on theoretical frameworks from the social sciences in order to understand how these factors influence, for example, students' motivation, study practices, and learning results. In this paper we explore the computing education research literature to identify new theoretical constructs that have emerged from this research. We focus on empirical work in programming education that extends or adapts theories or instruments from the social sciences or that independently develops theories specific to programming. From an initial data set of more than 3800 papers published in the years 2010--2019, we identify 50 papers that present a range of domain-specific theoretical constructs addressing emotions, affect, beliefs, attitudes, and self-efficacy. They include 11 validated instruments and a number of statistical models, but also grounded theories and pedagogical models. We summarize the main results of many of these constructs and provide references for all of them. We also investigate how these constructs have informed further research by analysing over 850 papers that cite these 50 papers. We categorize the ways that theories can inform further research, and give examples of papers in each of these categories. Our findings indicate that among these categories, instruments have been most widely used in further research, thus affirming their value in the field.

Peer review is a mainstay of academic publication - indeed, it is the peer-review process that provides much of the publications' credibility. This working group is examining the ways peer review is used in various computing education venues and will use this examination to articulate community standards for peer review in this discipline.

Computing education research (CER) can provide us valuable support for developing educational practices in computer science. In this presentation, I discuss a selection of results and experiences, mostly gathered from PhD research in CER that has been carried out at Aalto University during the last 20 years. The main research context has been large-scale courses in programming as well as data structures and algorithms with many hundreds of students and scarce human teaching resources. Therefore, a lot of our work has been motivated by the need for building engaging online learning content and providing better feedback for students on their learning results and progress. These include research in algorithm and program visualization, interactive e-books, as well as challenges in addressing dropout problems and procrastination in programming courses.

In order to mature as a research field, computing education research (CER) seeks to build a better theoretical understanding of how students learn computing concepts and processes. Progress in this area depends on the development of computing-specific theories of learning to complement the general theoretical understanding of learning processes. In this paper we analyze the CER literature in three central publication venues -- ICER, ACM Transactions of Computing Education, and Computer Science Education -- over the period 2005--2015. Our findings identify new theoretical constructs of learning computing that have been published, and the research approaches that have been used in formulating these constructs. We identify 65 novel theoretical constructs in areas such as learning/understanding, learning behaviour/strategies, study choice/orientation, and performance/progression/retention. The most common research methods used to devise new constructs include grounded theory, phenomenography, and various statistical models. We further analyze how a number of these constructs, which arose in computing education, have been used in subsequent research, and present several examples to illustrate how theoretical constructs can guide and enrich further research. We discuss the implications for the whole field.

Code examples are commonly used learning resources that help students grasp various programming structures and concepts. However, example code usually requires explanations about what each line or part of the code does. Otherwise, students may find it difficult to follow an example. In this paper, we compare two types of code examples that use different techniques to describe important concepts in the code: annotated and animated examples. The former displays an explanation for a subset of lines in plain text, whereas the latter visualizes code execution. We studied the use and impact of these enhanced examples, provided as non-mandatory practice content, in three introductory Java courses. Our results suggest that animated examples are more engaging and have a positive impact on students' learning. As compared to annotated examples, students spent more time with animated examples and more likely completed them. Also, a positive relationship was found between the number of explored animated examples and the overall course grade.

We analyze the Computing Education Research (CER) literature to discover what theories, conceptual models and frameworks recent CER builds on. This gives rise to a broad understanding of the theoretical basis of CER that is useful for researchers working in that area, and has the potential to help CER develop its own identity as an independent field of study.

During the past two years, Massive Open Online Courses (MOOCs) have created wide interest in the academic world raising both enthusiasm for new opportunities for universities and many concerns for the future of university education. The discussion has mainly appeared in non-scientific forums, such as magazine articles, columns and blogs, making it difficult to judge wider opinions within academia. To collect more rigorous data we surveyed teachers, researchers, and academic managers on their opinions and experiences of MOOCs. In this paper, we present our analysis of responses from the computer science academic community (n=137). Their feelings about MOOCs are highly mixed. Content analysis of open-ended questions revealed that the most often mentioned positive aspects included affordances of MOOCs, freedom of time and location for studying, and the possibility to experience teaching from top-level international teachers/experts. The most common negative aspects included concerns about pedagogical designs of MOOCs, assessment practices, and lack of interaction with the teacher. About half the respondents claimed they had not changed their teaching as a result of MOOCs, a small number used MOOCs as learning resources and very few were engaging with MOOCs in any significant way.

In this paper, we present a novel method to support automated formative assessment of programming tasks. Whereas traditional automated assessment tools mainly apply textual comparison of test program output vs. model output, we present a method that applies graphical output in terms of QR (Quick Response) codes. In our approach, programming tasks are formulated in such a way that the correct output of the task is a QR code. The correctness of the solution can therefore be tested by scanning the QR code. Despite the simple form of the tasks, they can support automated assessment with instant feedback, multiple programming languages, and simple statistics of students' performance. In some cases, feedback about misconceptions can also be given automatically. Moreover, they fit well to game-like learning environments, because the output can be interpreted as an URL which can lead to new clues or puzzles in a game. This is very useful, for example, in alternate reality games.

This article is a survey of program visualization systems intended for teaching beginners about the runtime behavior of computer programs. Our focus is on generic systems that are capable of illustrating many kinds of programs and behaviors. We inclusively describe such systems from the last three decades and review findings from their empirical evaluations. A comparable review on the topic does not previously exist; ours is intended to serve as a reference for the creators, evaluators, and users of educational program visualization systems. Moreover, we revisit the issue of learner engagement which has been identified as a potentially key factor in the success of educational software visualization and summarize what little is known about engagement in the context of the generic program visualization systems for beginners that we have reviewed; a proposed refinement of the frameworks previously used by computing education researchers to rank types of learner engagement is a side product of this effort. Overall, our review illustrates that program visualization systems for beginners are often short-lived research prototypes that support the user-controlled viewing of program animations; a recent trend is to support more engaging modes of user interaction. The results of evaluations largely support the use of program visualization in introductory programming education, but research to date is insufficient for drawing more nuanced conclusions with respect to learner engagement. On the basis of our review, we identify interesting questions to answer for future research in relation to themes such as engagement, the authenticity of learning tasks, cognitive load, and the integration of program visualization into introductory programming pedagogy.

The process of solving a programming assignment is generally invisible to the teacher. We only see the end result and maybe a few snapshots along the way. In order to investigate this process with regard to Parsons problems, we used an online environment for Parsons problems in Python to record a detailed trace of all the interaction during the solving session. In these assignments, learners are to correctly order and indent a given set of code fragments in order to build a functioning program that meets the set requirements. We collected data from students of two programming courses and among other analyses present a visualization of the solution path as an interactive graph that can be used to explore such patterns and anomalies as backtracking and loops in the solution. The results provide insights into students' solving process for these types of problems and ideas on how to improve the assignment environment and its use in programming education.

Program comprehension (PC) is a research field that has been extensively studied from different points of view, including human program understanding and mental models, automated program understanding, etc. In this paper, we discuss algorithm recognition (AR) as a subfield of PC and explain their relationship. We present a method for automatic AR from Java source code. The method is based on static analysis of program code including various statistics of language constructs, software metrics, as well as analysis of roles of variables in the target program. In the first phase of the method, a number of different implementations of the supported algorithms are analyzed and stored in the knowledge base of the system as learning data, and in the second phase, previously unseen algorithms are recognized using this information. We have developed a prototype and successfully applied the method for recognition of sorting algorithms. This process is explained in the paper along with the experiment we have conducted to evaluate the performance of the method. Although the method, at its current state, is still sensitive to changes made to target algorithms, the encouraging results of the experiment demonstrate that it can be further developed to be used as a PC method in various applications, as an example, in automatic assessment tools to check the algorithms used by students, the functionality that is currently missing from these tools.

In this paper, we introduce a new way to categorise existing educational research making it possible to find new previously overlooked research topics. This novel categorisation system is based on the didactic foci of the research papers. Our categorisation scheme is not data driven as in previously published categorisation systems but is derived from the didactic triangle, which is a theoretical model describing the elements of a teaching-studying-learning processes.

This paper presents a preliminary analysis of research papers in computing education. While previous analysis has explored what research is being done in computing education, this project explores how that research is being done. We present our classification system, then the results of applying it to the papers from all five years of ICER. We find that this subset of computing education research has more in common with research in information systems than with that in computer science or software engineering; and that the papers published at ICER generally appear to conform to the specified ICER requirements.

What is Computing Education Research (CER), why are we doing this type of research, and what should the community achieve? As associate editors to this special edition we provide our perspectives and discuss how they have influenced the evolution of the Koli Calling International Conference on Computing Education Research over the last nine years. The two articles in the special issue have been selected from among the twenty eight conference contributions for their originality and contribution to CER in the areas the Koli Calling Conference aims to emphasize. We hope that you find them inspiring reading.

The focus of this paper is on science students (CS minors) in a CS1 Java course at a technical university. The following three questions are discussed: 1) Which programming related topics the students at the target CS1 course find difficult to learn, and what is the difference between the students that passed the course and the students that dropped out of the course. 2) What kind of strategies both the passed and the dropout students used when they faced a difficult programming related topic. 3) Why some students decided to drop out of the course.

This paper introduces SEEQ - a System for Enhanced Exploration and Querying. It is a visual query system for databases that uses a diagrammatic visualization for most user interaction. The database schema is displayed as a graph of the data model including classes, associations and attributes. The user formulates the query in terms of direct manipulation as a graph of the schema objects, additional operators and constants. The output of the query is visualized as a graph of instances and constants or in some other format that is appropriate for the data. SEEQ can operate on arbitrary relational data base provided that the schema is in XML format.

This study focuses on CS minor students' decisions to drop out from the CS1 course. The high level of drop out percentage has been a problem at Helsinki University of Technology for many years. This course has yearly enrolment of 500-600 students and the drop out percentage has varied from 30-50 percents.Since we did not have clear picture of drop out reasons we conducted a qualitative interview research in which 18 dropouts from the CS1 course were interviewed. The reasons of drop out were categorized and, in addition, each case was investigated individually. This procedure enabled us to both list the reasons and to reveal the cumulative nature of drop out reasons.The results indicate that several reasons affect students' decision to quit the CS1 course. The most frequent reasons were the lack of time and the lack of motivation. However, both of these reasons were in turn affected by factors, such as the perceived difficulty of the course, general difficulties with time managing and planning studies, or the decision to prefer something else. Furthermore, low comfort level and plagiarism played a role in drop out. In addition, drop out reasons cumulated.This study shows that the complexity and large variety of factors involved in students' decision to drop the course. This indicates that simple actions to improve teaching or organization on a CS1 course to reduce drop out may be ineffective. Efficient intervention to the problem apparently requires a combination of many different actions that take into consideration the versatile nature of reasons involved in drop out.

In this paper, we present our experiences in using two automatic assessment tools, TRAKLA and TRAKLA2, in a second course of programming. In this course, 500--700 students have been enrolled annually during the period 1993--2004. The tools are specifically designed for assessing algorithm simulation exercises in which students simulate the working of algorithms at a conceptual level. Both of these tools allow students to resubmit their solutions after getting feedback. However, the resubmission policy has changed considerably during the period. Those changes reflect the students performance in the exercises. We conclude that an encouraging grading policy, i.e., the more exercises they solve the better grades they achive, combined with an option to resubmit the solution is a very important factor promoting students' learning. However, in order to prevent aimless trial-and-error problem solving method, the number of resubmissions allowed per assignment should be carefully controlled.

Software development is prone to time-consuming and expensive errors. Finding and correcting errors in a program (debugging) is usually done by executing the program with different inputs and examining its intermediate and/or final results (testing). The tools that are currently available for debugging (debuggers) do not fully make use of several potentially useful visualisation and interaction techniques.This article presents a prototype debugging tool (MVT--Matrix Visual Tester) based on a new interactive graphical software testing methodology called visual testing. A programmer can use a visual testing tool to examine and manipulate a running program and its data structures. The tool combines aspects of visual algorithm simulation, high-level data visualisation and visual debugging, and allows easier testing, debugging and understanding of software.

Algorithm animation has been researched since early 1980's and many different visualization systems have been developed. However, most of them have remained as research prototypes and almost none have gained wide acceptance by teachers as classroom demonstration tools. One of the key reasons for this has been that preparing animations has been too laborious. In this paper, we demonstrate a new tool, MatrixPro, in which animations are generated in terms of visual algorithm simulation. The user can graphically invoke ready-made operations available in the library to simulate the working of real algorithms. Since the system understands the semantics of the operations, the teacher can ex tempore demonstrate the execution of algorithms with different input sets, or work with "what-if" questions students ask in lectures. Such an approach lowers considerably the step for adopting algorithm visualization as a regular lecture tool.

Feedback based on automatic assessment of students' solutions is an important aid for students' learning process in self-study and distance learning. Most automatic assessment systems allow students to revise their solutions after getting the feedback and resubmit their work to be able to complete the exercise. In this paper, we analyze the effect of re submission in detail in the context of automatically assessed algorithm simulation exercises. In the target system TRAKLA2, students can revise their answers as many times as they wish, but each trial requires to restart the exercise with new random data. We present statistical results from a course with 600 students and show that our method that combines resubmissions and exercises with randomized initial data has a positive effect on learning results.

Data structures and algorithms include abstract concepts and processes, which people often find difficult to understand. Examples of these are complex data types and procedural encoding of algorithms. Software visualization can significantly help in solving the problem.

Several Web-based learning environments which can automatically give immediate feedback to the students have been reported within the past few years. The quality of feedback can be relatively high in these systems, but it does not achieve the level a trained teacher can provide. However, the lack of the best possible feedback can be compensated for, to some extent, by providing immediate and individualised feedback at any place or time. The question is whether the systems providing automatic feedback are good enough to compete with humans. This paper reports on a randomised large scale intervention study. We found that there was no significant difference in the final examination results between students doing instructed simulation exercises in a classroom session and students using a web-based learning environment, if the exercises were the same. However, with more challenging exercises, there was a significant difference in the examination results, while the drop out rate was higher. Thus, the chosen teaching method and medium did not effect the level of learning, but the quality of the exercises did.

In this paper, we present some experiences on using automatic assessment in large scale courses of introductory programming, data structures, and algorithms over a period of 10 years. Automatic assessment provides an effective method for giving immediate 24/7 feedback service for students of mass courses. A very important factor in the promoting of learning is the possibility to resubmit answers after receiving the feedback. However, our experience shows that the resubmission option is not the only key motivation factor. More important factors include the challenge of exercises and the grading scale or the course assignments. A successful combination of all of these can produce good learning results.

No abstract available.

Automatic assessment of programming exercises has become an important method for grading students' exercises and giving feedback for them in mass courses. We describe a system called Scheme-robo, which has been designed for assessing programming exercises written in the functional programming language Scheme. The system assesses individual procedures instead of complete programs. In addition to checking the correctness of students' solutions the system provides many different tools for analysing other things in the program like its structure and running time, and possible plagiarism. The system has been in production use on our introductory programming course with some 300 students for two years with good results.

Visualization is a useful aid for understanding the working of algorithms. Therefore many interactive algorithm animation tools have been developed. However, students may misinterpret the visualization and therefore the correctness of their interpretation should be confirmed by tests supplemented with feedback.In this paper, a learning environment for data structures and algorithms is presented. The combination of algorithm animation and simulation with automatic assessment provides a way to give meaningful feedback to the students. Our experience shows that this combination is of great value for the students studying algorithms.

Article Group updates for relaxed height-balanced trees Share on Authors: Lauri Malmi Laboratory of Information Processing Science, Helsinki University of Technology, P.O. Box 5400, 02015 HUT, Espoo, Finland Laboratory of Information Processing Science, Helsinki University of Technology, P.O. Box 5400, 02015 HUT, Espoo, FinlandView Profile , Eljas Soisalon-Soininen Laboratory of Information Processing Science, Helsinki University of Technology, P.O. Box 5400, 02015 HUT, Espoo, Finland Laboratory of Information Processing Science, Helsinki University of Technology, P.O. Box 5400, 02015 HUT, Espoo, FinlandView Profile Authors Info & Claims PODS '99: Proceedings of the eighteenth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systemsMay 1999 Pages 358–367https://doi.org/10.1145/303976.304011Online:01 May 1999Publication History 13citation341DownloadsMetricsTotal Citations13Total Downloads341Last 12 Months2Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access