Code quality is an important aspect of programming, as quality code is easier to maintain, and code maintenance makes up the majority of software cost. For that reason, code quality should be emphasized in programming education. Previous work has identified many code quality defects commonly made by students. However, the current state lacks a clear organization and prioritization of these defects. In this paper, we propose an organization framework for code quality defects, presenting a catalog that describes 80 defects, with a specific focus on defects frequently encountered in code by novice programmers. To determine which defects are worth pointing out to students, we conducted a survey among 72 educators, who rated the priority with which each defect should be reported to a student. These presented results serve multiple purposes: they facilitate comparison across various research studies, support the advancement of software tools, and offer inspiration for programming education.

Abstract Personalization in online learning environments has been extensively studied at various levels, ranging from adaptive hints during task-solving to recommending whole courses. In this study, we focus on recommending learning activities (sequences of homogeneous tasks). We argue that this is an important yet insufficiently explored area, particularly when considering the requirements of large-scale online learning environments used in practice. To address this gap, we propose a modular rule-based framework for recommendations and thoroughly explain the rationale behind the proposal. We also discuss a specific application of the framework.

To provide practice and assessment of computational thinking, we need specific problems students can solve. There are many such problems, but they are hard to find. Learning environments and assessments often use only specific types of problems and thus do not cover computational thinking in its whole scope. We provide an extensive catalog of well-structured computational thinking problem sets together with a systematic encoding of their features. Based on this encoding, we propose a four-level taxonomy that provides an organization of a wide variety of problems. The catalog, taxonomy, and problem features are useful for content authors, designers of learning environments, and researchers studying computational thinking.

Research on feedback in introductory programming focuses mostly on incomplete and incorrect programs. However, most of the functionally correct programs also contain defects that call for feedback. We analyzed 114,000 solutions to 161 short coding problems in Python and compiled a catalog of 32 defects in code quality. We found that most correct solutions contain some defects and that students do not stop making them if they do not receive targeted feedback. The catalog of defects, together with their prevalence across common topics like expressions, loops, and lists, informs educators which defects to address in which lectures and guides the development of exercises on code quality. Additionally, we describe defect detectors, which can be used to generate valuable feedback to students automatically.

Teaching assistants (TAs) are heavily used in computer science courses as a way to handle high enrollment and still being able to offer students individual tutoring and detailed assessments. TAs are themselves students who take on this additional role in parallel with their own studies at the same institution. Previous research has shown that being a TA can be challenging but has mainly been conducted on TAs from a single institution or within a single course. This paper offers a multi-institutional, multi-national perspective of challenges that TAs in computer science face. This has been done by conducting a thematic analysis of 180 reflective essays written by TAs from three institutions across Europe. The thematic analysis resulted in five main challenges: becoming a professional TA, student focused challenges, assessment, defining and using best practice, and threats to best practice. In addition, these challenges were all identified within the essays from all three institutions, indicating that the identified challenges are not particularly context-dependent. Based on these findings, we also outline implications for educators involved in TA training and coordinators of computer science courses with TAs.

Student models are typically evaluated through predicting the correctness of the next answer. This approach is insufficient in the problem-solving context, especially for student models that use performance data beyond binary correctness. We propose more comprehensive methods for validating student models and illustrate them in the context of introductory programming. We demonstrate the insufficiency of the next answer correctness prediction task, as it is neither able to reveal low validity of student models that use just binary correctness, nor does it show increased validity of models that use other performance data. The key message is that the prevalent usage of the next answer correctness for validating student models and binary correctness as the only input to the models is not always warranted and limits the progress in learning analytics.

Additive Factors Model is a widely used student model, which is primarily used for refining knowledge component models (Q-matrices). We explore the robustness and generalizability of the model. We explicitly formulate simplifying assumptions that the model makes and we discuss methods for visualizing learning curves based on the model. We also report on an application of the model to data from a learning system for introductory programming; these experiments illustrate possibly misleading interpretation of model results due to differences in item difficulty. Overall, our results show that greater care has to be taken in the application of the model and in the interpretation of results obtained with the model.

Quantification of the difficulty of problem solving tasks has many applications in the development of adaptive learning systems, e.g., task sequencing, student modeling, and insight for content authors. There are, however, many potential conceptualizations and measures of problem difficulty and the computation of difficulty measures is influenced by biases in data collection. In this work, we explore difficulty measures for introductory programming tasks. The results provide insight into non-trivial behavior of even simple difficulty measures.

Large scale learning systems for introductory programming need to be able to automatically assess the quality of students' performance on programming tasks. This assessment is done using a performance measure, which provides feedback to students and teachers, and an input to the domain, student and tutor models. The choice of a good performance measure is nontrivial, since the performance of students can be measured in many ways, and the design of measure can interact with the adaptive features of a learning system or imperfections in the used domain model. We discuss the important design decisions and illustrate the process of an iterative design and evaluation of a performance measure in a case study.

Block-based environments are today commonly used for introductory programming activities like those that are part of the Hour of Code campaign, which reaches millions of students. These activities typically consist of a static series of problems. Our aim is to make this type of activities more efficient by incorporating adaptive behavior. In this work, we discuss steps towards this goal, specifically a proposal and implementation of a programming game that supports both elementary problems and interesting programming challenges and thus provides an environment for meaningful adaptation. We also discuss methods of adaptivity and the issue of evaluating student performance while solving a problem.

A personalized learning system needs a large pool of items for learners to solve. When working with a large pool of items, it is useful to measure the similarity of items. We outline a general approach to measuring the similarity of items and discuss specific measures for items used in introductory programming. Evaluation of quality of similarity measures is difficult. To this end, we propose an evaluation approach utilizing three levels of abstraction. We illustrate our approach to measuring similarity and provide evaluation using items from three diverse programming environments.