Automated feedback generation plays a crucial role in enhancing personalized learning experiences in computer science education. Among different types of feedback, next-step hint feedback is particularly important, as it provides students with actionable steps to progress towards solving programming tasks. This study investigates how students interact with an AI-driven next-step hint system in an in-IDE learning environment. We gathered and analyzed a dataset from 34 students solving Kotlin tasks, containing detailed hint interaction logs. We applied process mining techniques and identified 16 common interaction scenarios. Semi-structured interviews with 6 students revealed strategies for managing unhelpful hints, such as adapting partial hints or modifying code to generate variations of the same hint. These findings, combined with our publicly available dataset, offer valuable opportunities for future research and provide key insights into student behavior, helping improve hint design for enhanced learning support.

Teaching the software engineers of the future to write high-quality code with good style and structure is important. This systematic literature review identifies existing instructional approaches, their objectives, and the strategies used for measuring their effectiveness. Building on an existing mapping study of code quality in education, we identified 53 papers on code structure instruction. We classified these studies into three categories: (1) studies focused on developing or evaluating automated tools and their usage (e.g., code analyzers, tutors, and refactoring tools), (2) studies discussing other instructional materials, such as learning resources (e.g., refactoring lessons and activities), rubrics, and catalogs of violations, and (3) studies discussing how to integrate code structure into the curriculum through a holistic approach to course design to support code quality. While most approaches use analyzers that point students to problems in their code, incorporating these tools into classrooms is not straightforward. Combined with further research on code structure instruction in the classroom, we call for more studies on effectiveness. Over 40% of instructional studies had no evaluation. Many studies show promise for their interventions by demonstrating improvement in student performance (e.g., reduced violations in student code when using the intervention compared with code that was written without access to the intervention). These interventions warrant further investigation on learning, to see how students apply their knowledge after the instructional supports are removed.

Various studies have studied the impact of Generative AI on Computing Education. However, they have focused on the implications for novice programmers. In this experience report, we analyze the use of GenAI as a support tool for learning, creativity, and productivity in a web development course for undergraduate students with extensive programming experience. We collected diverse data (assignments, reflections, logs, and a survey) and found that students used GenAI on different tasks (code generation, idea generation, etc.) with a reported increase in learning and productivity. However, they are concerned about over-reliance and incorrect solutions and want more training in prompting strategies.

Collecting data from students solving programming tasks is valuable for both researchers and educators. Such data can be used to analyze student behavior, identify errors and misconceptions, and for many other purposes. In this work, we propose KOALA, a configurable tool to collect student data using JetBrains IDEs. This tool collects code snapshots and IDE interactions, converts them into the ProgSnap2 format, and provides visualization analysis.

LLMs are increasingly used in programming education. However, little research has explored their use in teaching and learning code refactoring. In this study, we use a grounded-theory approach to examine student-LLM conversations during code refactoring exercises. Our preliminary results show that students use LLM in various modes, such as requesting a refactoring for the entire program at once or discussing refactoring possibilities in long conversations.

The in-IDE learning format represents a novel way of teaching programming to students entirely within an industry-grade IDE, allowing them to learn both the language and the necessary tooling at the same time. In this tutorial, we will teach the audience everything they need to know to create in-IDE courses and analyze how the students are working in them. In the first part of the tutorial, the audience will get to know the JetBrains Academy plugin that allows creating courses for IntelliJ-based IDEs such as IntelliJ IDEA and PyCharm. The participants will develop their own simple courses with theory, programming tasks, and quizzes, as well as employ some LLM-based features like automatic test generation. In the second part, we will learn how to use another plugin to collect code snapshots and the usage of IDE features of students when they are solving the tasks. Finally, the participants will solve tasks in their own course while using the data gathering plugin, and we will show them how to process and analyze the collected data. As the outcome of the tutorial, the audience will know how to create in-IDE courses, track the students' performance and analyze it, and will already have their own simple course and a dataset that can be expanded or used for further research.

A recent ITiCSE working group investigated when and how experts give feedback and hints at steps novice programmers take when solving programming problems. Based on the feedback literature and an analysis of expert feedback on steps, the working group designed guidelines for when and how to give feedback. The feedback provided by educators using these guidelines on a number of sequences of student steps varied a lot. In this paper, we try to answer the question of why educators give feedback at particular steps to novice learners of programming. We prepared six authentic sequences of student steps when solving an introductory programming task. The preprocessed sequences were used in a survey to gather information about when and why an expert would give feedback. Respondents annotated each step from one sequence with if and why they would give feedback at that step. Our survey received 47 responses. We qualitatively analyzed the responses, resulting in a coding scheme consisting of 19 different reasons for why experts intervene (or not) when novice learners work on introductory programming tasks. We found a considerable variety of reasons experts give for when and how to help students with feedback and hints. Also, sometimes one expert uses a reason at a step to explain why they do intervene, and another expert uses the same reason at the step to not intervene. The categories of experts' feedback indicators will pave the way for several future studies and applications, including learning systems trying to resemble expert feedback strategies.

Ever since the emergence of large language models (LLMs) and related applications, such as ChatGPT, its performance and error analysis for programming tasks have been subject to research. In this work-in-progress paper, we explore the potential of such LLMs for computing educators and learners, as we analyze the feedback it generates to a given input containing program code. In particular, we aim at (1) exploring how an LLM like ChatGPT responds to students seeking help with their introductory programming tasks, and (2) identifying feedback types in its responses. To achieve these goals, we used students' programming sequences from a dataset gathered within a CS1 course as input for ChatGPT along with questions required to elicit feedback and correct solutions. The results show that ChatGPT performs reasonably well for some of the introductory programming tasks and student errors, which means that students can potentially benefit. However, educators should provide guidance on how to use the provided feedback, as it can contain misleading information for novices.

In this work, we developed an algorithm for detecting code quality issues in the templates of online programming tasks, validated it, and conducted an empirical study on the dataset of student solutions. The algorithm consists of analyzing recurring unfixed issues in solutions of different students, matching them with the code of the template, and then filtering the results. Our manual validation on a subset of tasks demonstrated a precision of 80.8% and a recall of 73.3%. We used the algorithm on 415 Java tasks from the JetBrains Academy platform and discovered that as much as 14.7% of tasks have at least one issue in their template, thus making it harder for students to learn good code quality practices. We describe our results in detail, provide several motivating examples and specific cases, and share the feedback of the developers of the platform, who fixed 51 issues based on the output of our approach.

While functionality and correctness of code has traditionally been the main focus of computing educators, quality aspects of code are getting increasingly more attention. High-quality code contributes to the maintainability of software systems, and should therefore be a central aspect of computing education. We have conducted a systematic mapping study to give a broad overview of the research conducted in the field of code quality in an educational context. The study investigates paper characteristics, topics, research methods, and the targeted programming languages. We found 195 publications (1976-2022) on the topic in multiple databases, which we systematically coded to answer the research questions. This paper reports on the results and identifies developments, trends, and new opportunities for research in the field of code quality in computing education.

Teaching students to develop code of good quality is important. Refactoring -- rewriting a program into a semantically equivalent program of better quality -- is a common technique to improve code quality. It is therefore relevant for students to learn about refactoring, even for the smaller programs they write as beginners. However, students make mistakes when refactoring programs. Some of these mistakes appear often, and might be caused by misconceptions they have. In this paper, we investigate common student code refactoring misconceptions. We do this by analyzing log data containing program snapshots of students working on refactoring exercises in a tutoring system. We manually inspect all transitions from a correct program state to an incorrect state. We then use grounded theory to identify and categorize misconceptions students might have when refactoring programs. As a result, this work (1) defines the concept of refactoring misconception, and (2) provides an initial list of 25 such misconceptions, together with an accompanying website with full details.

The recent advent of highly accurate and scalable large language models (LLMs) has taken the world by storm. From art to essays to computer code, LLMs are producing novel content that until recently was thought only humans could produce. Recent work in computing education has sought to understand the capabilities of LLMs for solving tasks such as writing code, explaining code, creating novel coding assignments, interpreting programming error messages, and more. However, these technologies continue to evolve at an astonishing rate leaving educators little time to adapt. This working group seeks to document the state-of-the-art for code generation LLMs, detail current opportunities and challenges related to their use, and present actionable approaches to integrating them into computing curricula.

Abstract There are many aspects of code quality, some of which are difficult to capture or to measure. Despite the importance of software quality, there is a lack of commonly accepted measures or indicators for code quality that can be linked to quality attributes. We investigate software developers’ perceptions of source code quality and the practices they recommend to achieve these qualities. We analyze data from semi-structured interviews with 34 professional software developers, programming teachers and students from Europe and the U.S. For the interviews, participants were asked to bring code examples to exemplify what they consider good and bad code, respectively. Readability and structure were used most commonly as defining properties for quality code. Together with documentation, they were also suggested as the most common target properties for quality improvement. When discussing actual code, developers focused on structure, comprehensibility and readability as quality properties. When analyzing relationships between properties, the most commonly talked about target property was comprehensibility. Documentation, structure and readability were named most frequently as source properties to achieve good comprehensibility. Some of the most important source code properties contributing to code quality as perceived by developers lack clear definitions and are difficult to capture. More research is therefore necessary to measure the structure, comprehensibility and readability of code in ways that matter for developers and to relate these measures of code structure, comprehensibility and readability to common software quality attributes.

Every year, millions of students learn how to write programs. Learning activities for beginners almost always include programming tasks that require a student to write a program to solve a particular problem. When learning how to solve such a task, many students need feedback on their previous actions, and hints on how to proceed. In the case of programming, the feedback should take the steps a student has taken towards implementing a solution into account, and the hints should help a student to complete or improve a possibly partial solution. Only a limited number of learning environments for programming give feedback and hints on intermediate steps students take towards a solution, and little is known about the quality of the feedback provided. To determine the quality of feedback of such tools and to help further developing them, we create and curate data sets that show what kinds of steps students take when solving programming exercises for beginners, and what kind of feedback and hints should be provided. This working group aims to 1) select or create several data sets with steps students take to solve programming tasks, 2) introduce a method to annotate students' steps in these data sets, 3) attach feedback and hints to these steps, 4) set up a method to utilize these data sets in various learning environments for programming, and 5) analyse the quality of hints and feedback in these learning environments.

In the last few decades, numerous tutoring systems and assessment tools have been developed to support students with learning programming, giving hints on correcting errors, showing which test cases do not succeed, and grading their overall solutions. The focus has been less on helping students write code with good style and quality. There are several professional tools that can help, but they are not targeted at novice programmers.

Code quality has been receiving less attention than program correctness in both the practice of and research into programming education. Writing poor quality code might be a sign of carelessness, or not fully understanding programming concepts and language constructs. Teachers play an important role in addressing quality issues, and encouraging students to write better code as early as possible.

Formative feedback, aimed at helping students to improve their work, is an important factor in learning. Many tools that offer programming exercises provide automated feedback on student solutions. We have performed a systematic literature review to find out what kind of feedback is provided, which techniques are used to generate the feedback, how adaptable the feedback is, and how these tools are evaluated. We have designed a labelling to classify the tools, and use Narciss’ feedback content categories to classify feedback messages. We report on the results of coding a total of 101 tools. We have found that feedback mostly focuses on identifying mistakes and less on fixing problems and taking a next step. Furthermore, teachers cannot easily adapt tools to their own needs. However, the diversity of feedback types has increased over the past decades and new techniques are being applied to generate feedback that is increasingly helpful for students.

Code quality is a key issue in software development. The ability to develop software of high quality is therefore a key learning goal of computing programs. However, there are no universally accepted measures to assess the quality of code and current standards are consideredweak. Furthermore, there are many facets to code quality. Defining and explaining the concept of code quality is therefore a challenge faced by many educators. In this working group, we investigate the perceptions of code quality of students, teachers, and professional programmers. In particular, we are interested in the differences in views of code quality by students, educators, and professional programmers and which quality aspects they consider as more or less important. Furthermore, we are interested in which sources of information on code quality and its assessment are used by these groups. Eventually, this will help us to develop resources that can be used to broaden students' views on software quality.

Because low quality code can cause serious problems in software systems, students learning to program should pay attention to code quality early. Although many studies have investigated mistakes that students make during programming, we do not know much about the quality of their code. This study examines the presence of quality issues related to program flow, choice of programming constructs and functions, clarity of expressions, decomposition and modularization in a large set of student Java programs. We investigated which issues occur most frequently, if students are able to solve these issues over time and if the use of code analysis tools has an effect on issue occurrence. We found that students hardly fix issues, in particular issues related to modularization, and that the use of tooling does not have much effect on the occurrence of issues.

Educational research has established that learning can be defined as an enduring change in behaviour, which results from practice or other forms of experience. In introductory programming courses, proficiency is typically approximated through relatively small but frequent assignments and tests. Scaling these assessments to track significant behavioural change is challenging due to the subtle and complex metrics that must be collected from large student populations. Based on a four-semester study, we present an analysis of learning tool interaction data collected from 514 students and 38,796 solutions to practice programming exercises. We first evaluate the effectiveness of measuring workflow patterns to detect students at-risk of failure within the first three weeks of the semester. Our early predictor analysis accurately detects 81% of the students who struggle throughout the course. However, our early predictor also captures transient struggling, as 43% of the students who ultimately did well in the course were classified as at-risk. In order to better differentiate sustained versus transient struggling, we further propose a trajectory metric which measures changes in programming behaviour. The trajectory metric detects 70% of the students who exhibit sustained struggling, and mis-classifies only 11% of students who go on to succeed in the course. Overall, our results show how detecting changes in programming behaviour can help us differentiate between learning and struggling in CS1.

Formative feedback, aimed at helping students to improve their work, is an important factor in learning. Many tools that offer programming exercises provide automated feedback on student solutions. We are performing a systematic literature review to find out what kind of feedback is provided, which techniques are used to generate the feedback, how adaptable the feedback is, and how these tools are evaluated. We have designed a labelling to classify the tools, and use Narciss' feedback content categories to classify feedback messages. We report on the results of the first iteration of our search in which we coded 69 tools. We have found that tools do not often give feedback on fixing problems and taking a next step, and that teachers cannot easily adapt tools to their own needs.