Code reading and comprehension skills are essential for novices learning programming, and explain-in-plain-English tasks (EiPE) are a well-established approach for assessing these skills. However, manual grading of EiPE tasks is time-consuming and this has limited their use in practice. To address this, we explore an approach where students explain code samples to a large language model (LLM) which generates code based on their explanations. This generated code is then evaluated using test suites, and shown to students along with the test results. We are interested in understanding how automated formative feedback from an LLM guides students' subsequent prompts towards solving EiPE tasks. We analyzed 177 unique attempts on four EiPE exercises from 21 students, looking at what kinds of mistakes they made and how they fixed them. We found that when students made mistakes, they identified and corrected them using either a combination of the LLM-generated code and test case results, or they switched from describing the purpose of the code to describing the sample code line-by-line until the LLM-generated code exactly matched the obfuscated sample code. Our findings suggest both optimism and caution with the use of LLMs for unmonitored formative feedback. We identified false positive and negative cases, helpful variable naming, and clues of direct code recitation by students. For most students, this approach represents an efficient way to demonstrate and assess their code comprehension skills. However, we also found evidence of misconceptions being reinforced, suggesting the need for further work to identify and guide students more effectively.

Computer Science Curricula 2023 (CS2023) was released in early 2024 by the Joint Task Force of the ACM, IEEE-Computer Society, and AAAI. Viewed from an educational perspective, CS2023 reflects the decennial changes in computer science (CS) since the release of the previous curricular guidelines, Computer Science Curricula 2013. CS2023 comprises curricular content and curricular practices: the former includes updates to the computer science knowledge areas and a competency model framework while curricular practices cover other aspects such as program design and delivery issues.

Adversarial Thinking (AT) is essential in cybersecurity, fostering strategic problem-solving by anticipating worst-case scenarios.However, its integration into early computing education, especially in the first two years, remains underexplored.Introductory courses like CS 0 build foundational skills but are challenging to implement in resource-limited community colleges.Strengthening AT in these students can enhance their workforce readiness and support transfers to four-year programs.Servingness describes how Hispanic-Serving Institutions (HSIs) go beyond merely enrolling Latinx students to address their specific cultural, academic, and social needs.It involves creating inclusive environments where students feel a strong sense of belonging, supported by culturally relevant practices, leadership opportunities, and community engagement.These environments generate feelings of being accepted, valued, and included within a community or group, also called a sense of belonging.This work presents research focused on the incorporation of AT within the first two years of computer science education at community colleges, in particular to a course CS 0. By examining the integration of AT principles in early coursework, the study identifies key characteristics and a sense of belonging that are instrumental in cultivating AT capabilities among students.The findings aim to offer actionable insights for educators in community colleges, enabling them to more effectively prepare students for the complexities of modern computing careers and to address the broader needs of the cybersecurity field.Through this targeted educational approach, students can develop a more robust understanding of adversarial strategies, enhancing their overall computational thinking and problem-solving skills.

Novice programmers often struggle through programming problem solving due to a lack of metacognitive awareness and strategies. Previous research has shown that novices can encounter multiple metacognitive difficulties while programming, such as forming incorrect conceptual models of the problem or having a false sense of progress after testing their solution. Novices are typically unaware of how these difficulties are hindering their progress. Meanwhile, many novices are now programming with generative AI (GenAI), which can provide complete solutions to most introductory programming problems, code suggestions, hints for next steps when stuck, and explain cryptic error messages. Its impact on novice metacognition has only started to be explored. Here we replicate a previous study that examined novice programming problem solving behavior and extend it by incorporating GenAI tools. Through 21 lab sessions consisting of participant observation, interview, and eye tracking, we explore how novices are coding with GenAI tools. Although 20 of 21 students completed the assigned programming problem, our findings show an unfortunate divide in the use of GenAI tools between students who did and did not struggle. Some students who did not struggle were able to use GenAI to accelerate, creating code they already intended to make, and were able to ignore unhelpful or incorrect inline code suggestions. But for students who struggled, our findings indicate that previously known metacognitive difficulties persist, and that GenAI unfortunately can compound them and even introduce new metacognitive difficulties. Furthermore, struggling students often expressed cognitive dissonance about their problem solving ability, thought they performed better than they did, and finished with an illusion of competence. Based on our observations from both groups, we propose ways to scaffold the novice GenAI experience and make suggestions for future work.

Model curricula for baccalaureate computer science (CS) have been published regularly from 1968 through 2013. In early 2021, the ACM, IEEE-Computer Society, and the Association for the Advancement of Artificial Intelligence (AAAI) constituted a task force to revise these curricula, which have now been released as Computer Science 2023 Curricula (CS2023). The CS2023 curricular guidelines inform educators and administrators on the what, why, and how to cover undergraduate CS over the next decade. Like past guidelines, CS2023 provides curricular content - a knowledge model largely backward compatible with CS2013, supplemented by a competency framework influenced by Computing Curricula 2020 (CC2020) - and complementary curricular practices, which include articles by international experts on program design and delivery. Ongoing drafts of CS2023 were disseminated via the CS2023 website, along with regular publications or presentations at various computing education venues.

Publisher Copyright: © 2024 Owner/Author.

The interaction of computing and society, the requirement for ethical development and use of computing technology, and the responsibility our profession has to society, have never been greater. This poster provides an in-depth picture of how the "Society, Ethics, and The Profession" (SEP) knowledge area (KA) in the CS2023 curricular guidelines was designed, spanning the evolution of the KA, and how the CS2023 SEP KA was designed so that future CS graduates will have the SEP knowledge, skills, dispositions, and competencies to effectively and responsibly carry out their work as professional computer scientists. The ultimate goal is to help educators in integrating these new guidelines into their curricula.

This ITiCSE Working Group (WG) has two goals: first, to increase awareness of computing education research in the African countries, and second, to create and strengthen connections between computing education researchers in African countries and those in the larger computing education research community.

In the rapidly evolving Generative AI (GenAI) landscape, source code and natural language are being mixed and used in new ways. This presents opportunities for rethinking teaching practice in Introductory Programming (CS1) courses that includes, but goes beyond, assessment. In this paper we examine the reasons why and how instructors who are early adopters of GenAI are using it in their teaching, and why others are not. We also explore the changes and adaptations that are currently being made to practice. This is achieved by synthesizing insights from several recent studies that have collected primary data from introductory programming instructors who are teaching with, considering teaching with, or actively not teaching with GenAI.

We explored how undergraduate introductory programming students naturalistically used generative AI to solve programming problems. We focused on the relationship between their use of AI to their self-regulation strategies, self-efficacy, and fear of failure in programming. In this repeated-measures, mixed-methods research, we examined students' patterns of using generative AI with qualitative student reflections and their self-regulation, self-efficacy, and fear of failure with quantitative instruments at multiple times throughout the semester. We also explored the relationships among these variables to learner characteristics, perceived usefulness of AI, and performance. Overall, our results suggest that student factors affect their baseline use of AI. In particular, students with higher self-efficacy, lower fear of failure, or higher prior grades tended to use AI less or later in the problem-solving process and rated it as less useful than others. Interestingly, we found no relationship between students' self-regulation strategies and their use of AI. Students who used AI less or later in problem-solving also had higher grades in the course, but this is most likely due to prior characteristics as our data do not suggest that this is a causal relationship.

This research has three prongs, with each comparing open- and closed-book exam questions across six years (2017-2023) in a final year undergraduate applied machine learning course. First, the authors evaluated the performance of numerous LLMs, compared to student performance, and comparing open and closed book exams. Second, at a micro level, the examination questions and categories for which LLMs were most and least effective were compared. This level of analysis is rarely if ever, discussed in the literature. The research finally investigates LLM detection techniques, specifically their efficacy in identifying replies created wholly by an LLM. It considers both raw LLM outputs and LLM outputs that have been tampered with by students, with an emphasis on academic integrity. This study is a staff-student research collaboration, featuring contributions from eight academic professionals and six students.

Large language models (LLMs) are revolutionizing the field of computing education with their powerful code-generating capabilities. Traditional pedagogical practices have focused on code writing tasks, but there is now a shift in importance towards reading, comprehending and evaluating LLM-generated code. Alongside this shift, an important new skill is emerging -- the ability to solve programming tasks by constructing good prompts for code-generating models. In this work we introduce a new type of programming exercise to hone this nascent skill: 'Prompt Problems'. Prompt Problems are designed to help students learn how to write effective prompts for AI code generators. A student solves a Prompt Problem by crafting a natural language prompt which, when provided as input to an LLM, outputs code that successfully solves a specified programming task. We also present a new web-based tool called Promptly which hosts a repository of Prompt Problems and supports the automated evaluation of prompt-generated code. We deploy Promptly in one CS1 and one CS2 course and describe our experiences, which include student perceptions of this new type of activity and their interactions with the tool. We find that students are enthusiastic about Prompt Problems, and appreciate how the problems engage their computational thinking skills and expose them to new programming constructs. We discuss ideas for the future development of new variations of Prompt Problems, and the need to carefully study their integration into classroom practice.

In a previous Birds of a Feather discussion, we delved into the nascent applications of generative AI, contemplating its potential and speculating on future trajectories. Since then, the landscape has continued to evolve revealing the capabilities and limitations of these models. Despite this progress, the computing education research community still faces uncertainty around pivotal aspects such as (1) academic integrity and assessments, (2) curricular adaptations, (3) pedagogical strategies, and (4) the competencies students require to instill responsible use of these tools. The goal of this Birds of a Feather discussion is to unravel these pressing and persistent issues with computing educators and researchers, fostering a collaborative exploration of strategies to navigate the educational implications of advancing generative AI technologies. Aligned with this goal of building an inclusive learning community, our BoF is led by globally distributed leaders to facilitate multiple coordinated discussions that can lead to a broader conversation about the role of LLMs in CS education.

We present results of an in-depth survey of nearly 100 introductory programming (CS1) instructors in 18 countries spanning six continents. Although CS1 is well studied, relatively few broadly-scoped studies have been conducted, and none prior have exceeded regional scale. In addition, CS1 is a notoriously fickle and often changing course, and many might find it beneficial to know what other instructors are doing across the globe; perhaps more so as we continue to understand the impact of the COVID-19 pandemic on computing education and as the effects of Generative AI take hold. Expanding upon several surveys conducted in Australasia, the UK, and Ireland, this survey facilitates a direct comparison of global trends in CS1. The survey goes beyond environmental factors such as languages used, and examines why CS1 instructors teach what they do, in the ways they do. In total the survey spans 84 institutions and 91 courses in which a total of over 40,000 students are enrolled.

Large language models (LLMs) have recently taken many fields, including computer science, by storm. Most recent work on LLMs in computing education has shown that they are capable of solving most introductory programming (CS1) exercises, exam questions, Parsons problems, and several other types of exercises and questions. Some work has investigated the ability of LLMs to solve CS2 problems as well. However, it remains unclear how well LLMs fare against more advanced upper-division coursework, such as proofs in algorithms courses. After all, while known to be proficient in many programming tasks, LLMs have been shown to have more difficulties in forming mathematical proofs.

Recent developments in deep learning have resulted in code-generation models that produce source code from natural language and code-based prompts with high accuracy. This is likely to have profound effects in the classroom, where novices learning to code can now use free tools to automatically suggest solutions to programming exercises and assignments. However, little is currently known about how novices interact with these tools in practice. We present the first study that observes students at the introductory level using one such code auto-generating tool, Github Copilot, on a typical introductory programming (CS1) assignment. Through observations and interviews we explore student perceptions of the benefits and pitfalls of this technology for learning, present new observed interaction patterns, and discuss cognitive and metacognitive difficulties faced by students. We consider design implications of these findings, specifically in terms of how tools like Copilot can better support and scaffold the novice programming experience.

Recent breakthroughs in deep learning have led to the emergence of generative AI models that exhibit extraordinary performance at producing human-like outputs. Using only simple input prompts, it is possible to generate novel text, images, video, music, and source code, as well as tackle tasks such as answering questions and translating and summarising text.

The recent advent of highly accurate and scalable large language models (LLMs) has taken the world by storm. From art to essays to computer code, LLMs are producing novel content that until recently was thought only humans could produce. Recent work in computing education has sought to understand the capabilities of LLMs for solving tasks such as writing code, explaining code, creating novel coding assignments, interpreting programming error messages, and more. However, these technologies continue to evolve at an astonishing rate leaving educators little time to adapt. This working group seeks to document the state-of-the-art for code generation LLMs, detail current opportunities and challenges related to their use, and present actionable approaches to integrating them into computing curricula.

The recent emergence of code generation tools powered by large language models has attracted wide attention. Models such as OpenAI Codex can take natural language problem descriptions as input and generate highly accurate source code solutions, with potentially significant implications for computing education. Given the many complexities that students face when learning to write code, they may quickly become reliant on such tools without properly understanding the underlying concepts. One popular approach for scaffolding the code writing process is to use Parsons problems, which present solution lines of code in a scrambled order. These remove the complexities of low-level syntax, and allow students to focus on algorithmic and design-level problem solving. It is unclear how well code generation models can be applied to solve Parsons problems, given the mechanics of these models and prior evidence that they underperform when problems include specific restrictions. In this paper, we explore the performance of the Codex model for solving Parsons problems over various prompt variations. Using a corpus of Parsons problems we sourced from the computing education literature, we find that Codex successfully reorders the problem blocks about half of the time, a much lower rate of success when compared to prior work on more free-form programming tasks. Regarding prompts, we find that small variations in prompting have a noticeable effect on model performance, although the effect is not as pronounced as between different problems.

When seeking to maximise the authenticity of assessment in programming courses it makes sense to provide students with practical programming problems to solve in an environment that is close to real software development practice, i.e., online, open book, and using their typical development environment. This creates an assessment environment that should afford students sufficient opportunities to evidence what they have learned, but also creates practical challenges in terms of academic integrity, flexibility in the automated grading process, and assumptions surrounding how the student may attempt to solve the problems both in terms of correct and incorrect solutions. In this experience report, we outline two independently observed cohorts of students sitting the same Java programming exam, with different weights, over three years. This is undertaken as a reflective exercise in order to derive a series of recommendations and retrospectively obvious pitfalls to act as guidance for educators considering online programming exams for large (i.e. n > 150) introductory programming courses. After discussing our assessment methodology, we provide 4 high-level observations and centre a set of recommendations around these to aid practitioners in their assessment design.

A key part of learning to program is learning to understand programming error messages. They can be hard to interpret and identifying the cause of errors can be time-consuming. One factor in this challenge is that the messages are typically intended for an audience that already knows how to program, or even for programming environments that then use the information to highlight areas in code. Researchers have been working on making these errors more novice friendly since the 1960s, however progress has been slow. The present work contributes to this stream of research by using large language models to enhance programming error messages with explanations of the errors and suggestions on how to fix them. Large language models can be used to create useful and novice-friendly enhancements to programming error messages that sometimes surpass the original programming error messages in interpretability and actionability. These results provide further evidence of the benefits of large language models for computing educators, highlighting their use in areas known to be challenging for students. We further discuss the benefits and downsides of large language models and highlight future streams of research for enhancing programming error messages.

The current ACM/IEEE/AAAI Computer Science Curricula is approaching ten years of age. The CS2023 Steering Committee began efforts to update this important document in spring 2021 (csed.acm.org). The aim of this session is to seek feedback from the community on an advanced draft of the Society, Ethics and Professionalism (SEP) knowledge area. Much has transpired in the last decade within these domains including the role of social media in our daily lives, professional roles, and even our elections. Today, fake news, data leaks, hacks, and scandals involving personal information are rife. The impact of computing on society has never been more high-stakes, our ethics have never been tested in these ways, and the need for up-to-date professionalism has never been greater. In this light we seek to gather as many diverse views as possible to help us shape a Society, Ethics & Professionalism knowledge area that can serve the community and our students for the next decade.

The introductory programming sequence has been the focus of much research in computing education. The recent advent of several viable and freely-available AI-driven code generation tools present several immediate opportunities and challenges in this domain. In this position paper we argue that the community needs to act quickly in deciding what possible opportunities can and should be leveraged and how, while also working on overcoming otherwise mitigating the possible challenges. Assuming that the effectiveness and proliferation of these tools will continue to progress rapidly, without quick, deliberate, and concerted efforts, educators will lose advantage in helping shape what opportunities come to be, and what challenges will endure. With this paper we aim to seed this discussion within the computing education community.

The introduction of Large Language Models (LLMs) has generated a significant amount of excitement both in industry and among researchers. Recently, tools that leverage LLMs have made their way into the classroom where they help students generate code and help instructors generate learning materials. There are likely many more uses of these tools -- both beneficial to learning and possibly detrimental to learning. To help ensure that these tools are used to enhance learning, educators need to not only be familiar with these tools, but with their use and potential misuse. The goal of this BoF is to raise awareness about LLMs and to build a learning community around their use in computing education. Aligned with this goal of building an inclusive learning community, our BoF is led by globally distributed discussion leaders, including undergraduate researchers, to facilitate multiple coordinated discussions that can lead to a broader conversation about the role of LLMs in CS education.

Reading a programming error message is the first step in understanding what it is trying to tell the programmer about how to fix an error in their code. However, these are often difficult to read, especially for novices which is not surprising given that error messages in many of the most popular languages in which novices learn to code were not written with readability in mind. As a result, novices frequently struggle to understand them. This is a long-standing problem, with researchers highlighting concerns about programming error message readability over the last six decades. Very recent work has put forward evidence of the need for measuring readability in error messages and a framework for doing so. This framework consists of four factors of readability for programming error messages: message length, vocabulary, jargon, and sentence construction. We use this framework to implement an approach to automatically assess the readability of programming error messages. Using established readability factors as predictors in a machine learning model, we train several models using a dataset of C and Java error messages. We examine the performance of these models, and apply the best performing model to a previously published set of messages evaluated for readability by experts, non-experts and students. Our results validate the previously proposed readability factors, and our model classifies messages similarly to human raters. Finally, we discuss future work needed to improve the accuracy of the model.

Recent developments, guidelines, and acts by the European Commission have started to frame policy for AI and related areas such as ML and data, not only for the broader community, but in the context of education specifically. This poster presents a succinct overview of these developments. Specifically, we look to bring together all publications that might impact the teaching of AI (for example, teacher expectations in the coming years around AI competencies)and publications that affect the use of AI in the classroom. We mean using tools and systems that incorporate both 'Good Old Fashioned 'AI and those that can directly impact students. This poster is of value to both the European and the wider CER communities and practitioners, as it brings together several guidelines, acts, and plans that are not easily searchable or linked. The publications presented in this poster will impact the teaching of AI and teaching with Ai in Europe, and insights can be drawn and compared for other jurisdictions as the educational world adapts to and with AI.

Programming error messages (PEMs) have long been a hindrance to novice programmers. This work aims to establish a catalog of PEM anti-patterns--- common, reoccurring features of PEMs that make them unhelpful or actively harmful to programmers. The goal is for educators to be aware of, and actively teach concrete ways that PEMs can be misleading to students; to encourage language implementers to be cognizant of these; and avoid them when designing error feedback. A pilot study is being conducted to validate the presence of anti-patterns in error messages.

Background and Context. Metacognitive skills are important for all students learning to program and interest in applying pedagogical approaches in early programming courses that focus on metacognitive aspects is growing. However, most studies of such approaches are not rigorously based in theory, and when they are, almost always utilize foundational education and psychology theories from as far back as the 1970s. More recent theory is less tested, and not all relevant metacognitive theories have been explored in the computing education research literature.

In 2021 CSINC (a TU Dublin research group) launched their online Computer Science MOOC called CSLINC(Computer Science Inclusive Learning Environment). CSLINC provides a suite of eight week courses in computer science (CS) which are scaffolded for teachers to support their delivery in the classroom Quille22. The uptake for CSLINC was significant with 230 teachers delivering material to 12,000 students. The goals of CSLINC are to attempt to change CS perceptions and increase the number of students choosing Leaving Certificate Computer Science (LCCS) and also to increase teacher programming self-efficacy in the delivery of CS materials, thus leading to more teachers considering to deliver the formal curricula. Previous research conducted in the area of teacher programming self-efficacy found that programming self-efficacy is lower for formal-curricula second-level CS teachers compared to CS1 Students Faherty21. In the study teachers were attending CPD aimed at those teaching or planning to teach the LCCS subject at upper second level. The purpose of this pilot study is to do an initial comparison between the programming self-efficacy of teachers who have used CSLINC (who do not deliver formal CS curricula) in the classroom compared to the teachers who are or were planning to deliver LCCS (formal curricula).

Humans adjust how they speak depending on context. Two key facets of this are utilizing different vocabulary and speaking rates depending on the audience. Exactly how we use language while teaching may depend on our students, their backgrounds and needs, and the subject matter. How we speak in the classroom likely affects student comprehension and may affect equity and accessibility.

In this paper, we analyze 32,000 Java programming assessments submitted to the CodeRunner platform by introductory programming students. We identify common sequences of errors and link these patterns with observations of code that compiles but still contains logical errors. We establish reoccurring errors and common error pathways through a Markov chain analysis of attempts and use Association Rule Mining to link common error patterns with logical errors that occur later in the attempts. As a result, we provide a number of suggestions for instructors of introductory Java courses regarding how to potentially navigate these issues in their teaching practice and discuss possible pedagogical interventions on the basis of our observations.

The session will present an overview of findings of a recently funded NSF workshop that set out to examine the pressing issues for computing education research for the next 15 years. Based on dialogs for scholars working in this area, the workshop participants developed a series of themes and topics that they felt should become the focus of computing education research efforts for the next 15 years. Main themes that emerged and will be discussed in this session include: diversity, equity, inclusion, ethics, broadening participation, teaching, learning, K-12, research to practice, computing's connection to other fields, and computing education research disciplinary issues. The session will focus on interesting research questions from each of these areas that are ripe to be explored as well as enablers and blockers to the progress of this work.

As computing students progress through their studies they become proficient with multiple programming languages. Prior work investigating language transitions for novices has tended to analyze program artifacts rather than explore the benefits and difficulties as perceived by students in their own words, and has often overlooked problems that may arise in switching paradigms or where familiar syntax has a different meaning in the new language. In this paper, we ask students to reflect on the transition from an interpreted language and environment (MATLAB) to a compiled language (C), prompting comments on the aspects of learning the new language that they found both easier and harder. Analysis of over 70,000 words written by 771 students revealed that the highest-performing students expressed more negative sentiments towards the language transition -- a surprising result that we hypothesize is explained by their generally stronger metacognitive skills. We also report the most common difficulties described by students, which include challenges with syntax, error messages, and the process of compilation, and suggest teaching practices that might help students as they transition to a new programming language.

In the computing education community, discussion is growing about the benefits of teaching programming by explicitly using notional machines to help students. To-date most work is largely theoretical and little work addresses actually using them in a classroom. This paper documents our experience of creating a notional machine for a specific course and using it in that classroom. A key point we learned while creating this notional machine is that many of the difficulties encountered were due to the concept of a notional machine being tightly coupled to students' mental models. Although not surprising, the numerous complications this brings are important to overcome. The potential amount of detail included in the notional machine is enormously influenced by the students' mental models, which are likely specific to a course, and also change throughout a semester -- and certainly across several semesters. We present lessons learned from this experience, among them that implementing a notional machine and using it in class is a non-trivial yet possibly beneficial exercise.

Previous studies investigating and identifying non-technical skills of computing show that creative skills play an important role in tackling difficult programming problems. In the field of cognitive psychology, creativity has been extensively researched, and many of these methods can be adapted for application in computing education. We conducted a systematic literature review to support research on creativity in computing education by summarizing relevant theories, instruments, and other prior work. The review encompasses all SIGCSE venues and major journals in the field, providing a perspective of the current landscape on teaching and assessing creativity, in the context of computing in higher education. We identify which papers explore creativity as a central theoretical basis, revealing eight major themes. We also found seven commonly used measurement instruments. Creativity theories that are notably absent are also discussed, as are pedagogical implications of the approaches to fostering creativity in computing education. This paper serves to support the community by highlighting the interdisciplinary aspects of creativity research applicable in computing contexts. Additionally, it provides practical guidance and implications for educators in leveraging creativity in the classroom.

In this work we investigate compilation behavior and error resolution time of thousands of novice programmers using two different versions of the BlueJ pedagogical Java programming environment. The two versions feature different compilation and error message presentation mechanisms. BlueJ 3 is a click-to-compile environment with enforced first error message presentation whereas BlueJ 4 features automatic compilation and on-demand error message presentation. We provide an overview of compilation activity based on novices' programming time spent in different compilation states and also present differences in the time it takes students to reach successful compilation states in each version. We find that compilation activity differs for each version and that students reach successful compilations faster in BlueJ 4. Based on our findings, we discuss the effectiveness of each version and argue against the constant red underlining of code - an error indicator mechanism present in BlueJ 4 - as this can lead to false positive errors that may mislead novices during their programming tasks.

We present an overview of a "Machine Learning and Artificial Intelligence" course that is part of a large online course platform for upper second level students. We take a novel approach to teaching fundamental AI concepts that does not require code, and assumes little prior knowledge including only basic mathematics. The design ethos is for students to gain an understanding of how algorithms can "learn". Many misconceptions exist about this term with respect to AI and can lead to confusion and more serious misconceptions, particularly for students who engage with AI-enabled tools regularly. This approach aims to provide insights into how AI actually works, to demystify and remove barriers to more advanced learning, and to emphasize the important roles of ethics and bias in AI. We took several steps to engage students, including videos narrated by a final-year second-level student (US 12th grade). We present design and logistics particulars on this course which is currently being taken by ~7,000 students in Ireland. We believe this will be of value to other educators and the wider community.

This poster introduces CSLINC, a free scaffolded MOOC framework tailored to second-level students in Ireland that consists of: an online platform built for accessibility; a suite of modules developed upon international best practices with varying co-creators; and automated assessment and certificates of completion. Its aim is to provide content to promote national CS curricula to all second-level students in Ireland. In September 2021, CSLINC launched to 10,000 students across 100 schools. Future work will include collecting and collating research to validate CSLINC's objectives, scaffolding that will build foundations for national curriculum learning outcomes, and measure its impact on students, their perceptions and follow on CS uptake at second-level in Ireland.

New trends in computer architecture include non-general purpose architectures, brain-inspired design, agile hardware development and environmentally responsible design. Computer Science students need to understand computer architecture to develop programs that can achieve high performance through a programmer's awareness of parallelism and latency. In a re-visit of curriculum guidelines, students could develop skills and competencies in less mature yet cutting-edge topics, including quantum computing. For example, should students understand and appreciate quantum processor design's components and characteristics? In a task to revise curricular guidelines, one faces the decision of which topics may be obsolete and should be dropped or modified. Equally important is deciding the set of topics that must be included. Such questions are hard to address, particularly for a curriculum intended to remain fresh under a 10-years horizon. For example, the danger of not properly promoting the discussion of quantum processor design in the new ACM/IEEE-CS/AAAI CS202X curricula is the long waiting cycle (15 years+) for any new revision. This BOF will foster the participants' interactions into a debate of whether new curricular guidelines should contain quantum processor design as part of the Architecture and Organization (AR) Knowledge Area.

In 2018 Ireland launched a new, nationwide Computer Science curriculum for upper second-level students. The biggest challenge preventing uptake of the subject is a low supply of teachers qualified to teach the subject resulting in most schools not offering the subject, preventing most students from having the choice in the first place. Currently, approximately 150 secondary schools (of 723 nationally) offer the subject. Most school teachers who aspire to teach CS do not have a degree in computing and typically have very low self-efficacy in programming, as previously reported in the literature. There is a need to help these teachers to confidently de-liver the national CS curriculum at second-level in Ireland. CSLINC is a platform and MOOC offering teachers a scaffolded approach towards building their confidence for formal CS content delivery. CSLINC provides courses that teachers can deliver to junior second-level students before the official, national CS curriculum is offered at senior-level. This poster provides overview of the benefits of this system for second-level teachers in Ireland, which will also be of value to the computing education community internationally.

As Computer Science Curricula 2013 (CS2013) approaches its 10th anniversary plans are underway to update this important document. The CS202X task force started these efforts in Spring 2021. The aim of this session is to seek feedback from the community on a draft of the Software Engineering (SE) knowledge area. As Dave Parnas once said, "Software Engineering is the multi-person construction of multi-version programs," it is distinct from programming by virtue of being acutely impacted by issues of time (compatibility, versioning, version skew, schema evolution) and coordination or teamwork (communication, coordination, planning, etc). These are challenging issues to convey effectively given classroom practicalities, but it is essential for our graduates that go on to industry roles to have some awareness of these matters. We hope that by hosting a BoF at SIGCSE we can gather input from current instructors, especially to hear what is and is not working well from the CS2013 guidelines. We also hope such a BoF session can serve as a networking session for educators and education-aligned industry representatives so that we have a better feedback loop as Software Engineering practices evolve.

Metacognition and self-regulation are important skills for successful learning and have been discussed and researched extensively in the general education literature for several decades. More recently, there has been growing interest in understanding how metacognitive and self-regulatory skills contribute to student success in the context of computing education. This article presents a thorough systematic review of metacognition and self-regulation work in the context of computer programming and an in-depth discussion of the theories that have been leveraged in some way. We also discuss several prominent metacognitive and self-regulation theories from the literature outside of computing education—for example, from psychology and education—that have yet to be applied in the context of programming education. In our investigation, we built a comprehensive corpus of papers on metacognition and self-regulation in programming education, and then employed backward snowballing to provide a deeper examination of foundational theories from outside computing education, some of which have been explored in programming education, and others that have yet to be but hold much promise. In addition, we make new observations about the way these theories are used by the computing education community, and present recommendations on how metacognition and self-regulation can help inform programming education in the future. In particular, we discuss exemplars of studies that have used existing theories to support their design and discussion of results as well as studies that have proposed their own metacognitive theories in the context of programming education. Readers will also find the article a useful resource for helping students in programming courses develop effective strategies for metacognition and self-regulation.

Programming error messages play an important role in learning to program. The cycle of program input and error message response completes a loop between the programmer and the compiler/interpreter and is a fundamental interaction between human and computer. However, error messages are notoriously problematic, especially for novices. Despite numerous guidelines citing the importance of message readability, there is little empirical research dedicated to understanding and assessing it. We report three related experiments investigating factors that influence programming error message readability. In the first two experiments we identify possible factors, and in the third we ask novice programmers to rate messages using scales derived from these factors. We find evidence that several key factors significantly affect message readability: message length, jargon use, sentence structure, and vocabulary. This provides novel empirical support for previously untested long-standing guidelines on message design, and informs future efforts to create readability metrics for programming error messages.

Aspects of computing are used in nearly all professions. Regardless of discipline, graduates require ever-improving computing competencies-applied in their own disciplinary contexts-to be proficient experts and informed citizens. However, teaching and learning contextually relevant computing competencies to non-computing students is fraught with challenges. One of those is the effective use of computing terminology. Terminology is fundamental to most human activities and is particularly critical where disciplines meet. We seek to investigate the nature of, and barriers presented by, computing terminology used in non-computing disciplines.

Many structural and social barriers influence what students study and their career trajectories. Computing is far from immune to such barriers, which can result in a recurring cycle of poor uptake, stagnating interest, the perpetuation of misconceptions, and a resulting lack of role models. We present 'Computing Crossroads', a project that, by investigating and highlighting career diversity, also reveals many other types of natural diversity in computing students and professionals. We describe our approach, progress to-date, and future plans, which we believe can serve to highlight the diversity that does exist in computing, and hopefully help lower barriers and improve diversity of all kinds.

Teacher self-esteem has been found to impact student learning in a number of non-computing fields. As computing slowly becomes a part of the upper secondary school (high school) curriculum in many countries, instruments designed to measure teachers' programming self-esteem can help inform classroom practice and processes such as teacher professional development needs. This study examines if there are differences in programming self-esteem (using the Bergin Programming Self-Esteem Instrument) between upper secondary school teachers and CS1 students in Ireland. In addition this study provides evidence of validity when using this instrument(originally developed for CS1 students) to measure upper secondary school teacher programming self-esteem. To test for evidence of validity, we compared the results of the programming self-esteem construct given to upper secondary school teachers (n=130) to a recent study of programming self-esteem among CS1 students (n=693). We found evidence of both reliability and validity with teachers that aligns with the evidence found for the CS1 students, demonstrating utility for use with teacher cohorts. Comparing these findings, teachers reported statistically significantly lower programming self-esteem compared to CS1 students. Interestingly CS1 students identifying as male had a statistically significant higher programming self-esteem than those identifying as female. However, we found no statistically significant difference for teacher gender, unlike previous work. Our results indicate that teacher programming self-esteem should be given consideration in the design and implementation of professional development.

This work focuses on the effect that programming time has on novice programmers' interaction with two versions of the BlueJ programming environment that differ in compilation mechanism and error message presentation. We utilize programming process data from users of both BlueJ versions, and those who used each one exclusively. Results indicate there is a threshold of approximately twenty hours of programming below and above which compilation and error message frequency changes. This indicates that being exposed to an environment for prolonged time can influence the programming interaction between novices and environments. This phenomenon is not yet understood.

Like many others, our institution had to adapt our traditional proctored, written examinations to open-book online variants due to theCOVID-19 pandemic. This paper describes the process applied to develop open-book online exams for final year (undergraduate)students studying Applied Machine Learning and Applied Artificial Intelligence and Deep Learning courses as part of a four-year BSc in Computer Science. We also present processes used to validate the examinations as well as plagiarism detection methods implemented. Findings from this study highlight positive effects of using open-book online exams, with ~85% of students reporting that they either prefer online open-book examinations or have no preference between traditional and open-book exams. There were no statistically significant differences reported comparing the exam results of student cohorts who took the open-book online examination, compared to previous cohorts who sat traditional exams. These results are of value to the CSEd community for three reasons. First, it outlines a methodology for developing online open-book exams(including publishing the open-book online exam papers as samples). Second, it provides approaches for deterring plagiarism and implementing plagiarism detection for open-book exams. Finally, we present feedback from students which may be used to guidefuture online open-book exam development.

The ACM Special Interest Group on Computer Science Education (SIGCSE) is one of the oldest and largest SIGs, and the SIGCSE Technical Symposium is the oldest and largest of the four SIGCSE conferences. However, the vast majority of Symposium attendees and contributors are from the United States. Because SIGCSE is an international organization, this lack of geographic diversity and representation is troubling because it may stifle collaboration, membership, professional development, and dissemination of research, and have many other adverse effects.

Sense of belonging, or belongingness, describes how accepted one feels in their academic community and is an important factor in creating inclusive learning environments.Belongingness is influenced by many factors including: students' backgrounds and experiences; other people; environments (physical and virtual); academic discipline; external factors such as local, regional, and global issues; and time.2020 has been dominated by several major events including the COVID-19 pandemic which dramatically impacted education.The Black Lives Matter movement has further raised global awareness of equality, diversity and inclusion not just in society, but in educational contexts.Climate change concerns, and politicallycharged news are also increasingly affecting our students.We have been monitoring our undergraduate computing students' sense of belonging for over three years, providing us with a unique opportunity to gauge recent changes during the pandemic.Our results surprised us.We found statistically significant reductions in the belongingness of students identifying as men as well as those not identifying as being part of a minority.However, investigating intersectionality of self-identified gender and minority status revealed more complicated and nuanced trends, illustrating important shifts in the belongingness of our students that we are only beginning to understand.

Metacognition and self-regulation are popular areas of interest in programming education, and they have been extensively researched outside of computing. While computing education researchers should draw upon this prior work, programming education is unique enough that we should explore the extent to which prior work applies to our context. The goal of this systematic review is to support research on metacognition and self-regulation in programming education by synthesizing relevant theories, measurements, and prior work on these topics. By reviewing papers that mention metacognition or self-regulation in the context of programming, we aim to provide a benchmark of our current progress towards understanding these topics and recommendations for future research. In our results, we discuss eight common theories that are widely used outside of computing education research, half of which are commonly used in computing education research. We also highlight 11 theories on related constructs (e.g., self-efficacy) that have been used successfully to understand programming education. Towards measuring metacognition and self-regulation in learners, we discuss seven instruments and protocols that have been used and highlight their strengths and weaknesses. To benchmark the current state of research, we examined papers that primarily studied metacognition and self-regulation in programming education and synthesize the reported interventions used and results from that research. While the primary intended contribution of this paper is to support research, readers will also learn about developing and supporting metacognition and self-regulation of students in programming courses.

It is well known that programming error messages can be notoriously difficult for novices to understand, hampering progress and leading to frustration. In response, researchers have explored various approaches for enhancing such messages, yet results from this active strand of research are currently mixed. Direct comparisons of results between studies is challenging as these typically investigate different kinds of message enhancements and report results using different metrics. In addition, many prior studies have involved code writing tasks. In such cases, not all students encounter the same errors and messages, and it is difficult to isolate the time spent interpreting messages and resolving errors from the time spent writing code. In this research, we explore the effects of presenting novices with compiler error messages designed using the most recent collection of published guidelines - specifically, more easily readable, short, positive messages containing resolution hints. To accurately determine the time and effort required to read and respond to the messages, we utilise a debugging task where all students are presented the same code and therefore encounter the same errors. We present results of a randomised controlled experiment (n > 700) which shows that, compared to standard error messages, the messages we tested resulted in significantly shorter debugging times and higher self-reported scores of message usefulness for students in the very early stages of learning a new language.

The term "the machine" is commonly used to refer to the complicated physical hardware running similarly complex software that ultimately executes programs. The idea that programmers write programs for a notional machine - an abstract model of an execution environment - not the machine itself, has risen to the point of gaining acceptance as a useful device in computing education. This has seeded a growing discussion about how explicitly utilizing notional machines in teaching can help students construct more accurate mental models, which is essential for learning programming. Much of the existing literature necessarily involves specific languages, visualization, and/or facilitating tools, and is not very accessible to many practitioners. Less focus has been put on how teachers can make explicit use of notional machines in their teaching. In this paper we describe notional machines and their use in a manner that is more accessible to a general educator audience in order to facilitate more effective computing education at all levels. We advocate explicitly delineating between visualization tools and the notional machines they depict, isolating and clarifying the notional machine so that it is conspicuous, apparent and useful. We present examples of how this approach can facilitate a more consistent method of teaching computing, and be used in more effective pedagogical practice for teaching computing.

Industry expectations of graduates are higher than ever. Not only are they required to be skilled in several technologies, but also need to be equipped with non-technical skills - often called soft skills or professional skills. This puts pressure on computing programs, as educators try to integrate these requirements into already full curricula. Although incorporating such skills into programs is seemingly common practice, little is known about what skills are being taught and why, outside of isolated case studies. In this work we ask: What non-technical skills are expected of undergraduate students according to computing programs? To answer this we manually curated 278 non-technical syllabi from 110 universities in 30 European countries. The most frequently identified skills are teamwork, ethics, written/oral communication, and presentation skills, while the development of one's own values, motivating others, creativity, and empathy feature least frequently. By providing a detailed analysis and an interactive website visualizing this data, we hope to aid the community in reviewing which non-technical skills are taught with an aim to teaching the right skills to the right students. This work sheds new light on what is expected of undergraduate computing students in terms of non-technical skills and identifies areas where more coverage might be needed.

Student sense of belonging has been shown to be associated with many attributes such as motivation and persistence. However, sense of belonging can show variations according to factors such as race and gender. In this study, we examine the relationship between undergraduate Computer Science students' participation in networking, outreach, and mentoring activities and their sense of belonging. Results reveal lower levels of sense of belonging in women and self-identified minorities. However, we observed a higher sense of belonging in female students who participated in networking, outreach, and mentoring activities.

This paper outlines the longitudinal development of a K-12 outreach model, to promote Computer Science in Ireland. Over a three-year period, it has been piloted to just under 9700 K-12 students from almost every county in Ireland. The model consists of a two-hour camp that introduces students to a range of Computer Science topics: addressing computing perceptions, introduction to coding and exploration of computational thinking. The model incorporates on-site school delivery and is available at no cost to any interested school across Ireland. The pilot study so far collected over 3400 surveys (pre- and post- outreach delivery). Schools from all over Ireland self-selected to participate, including male only, female only and mixed schools. The no-cost nature of the model meant schools deemed "disadvantaged", to private fee-paying schools participated. Initial findings are very positive, including the balance of male and female participants, where in the 2017-18 academic year it was 56:44 and in 2019-20 (to date), it is 35:65 respectively. Once the model is validated and tweaked (based on survey data), the model will be published (open access) for other institutions to implement the model locally. In addition, the authors intend to link schools (that the team have worked with over the three years) with local institutions, thus developing a sustainable ecosystem for the program to continue. This paper describes the model structure and outlines early findings.

In this paper, we introduce ProgSnap2, a standardized format for logging programming process data. ProgSnap2 is a tool for computing education researchers, with the goal of enabling collaboration by helping them to collect and share data, analysis code, and data-driven tools to support students. We give an overview of the format, including how events, event attributes, metadata, code snapshots and external resources are represented. We also present a case study to evaluate how ProgSnap2 can facilitate collaborative research. We investigated three metrics designed to quantify students' difficulty with compiler errors - the Error Quotient, Repeated Error Density and Watwin score - and compared their distributions and ability to predict students' performance. We analyzed five different ProgSnap2 datasets, spanning a variety of contexts and programming languages. We found that each error metric is mildly predictive of students' performance. We reflect on how the common data format allowed us to more easily investigate our research questions.

High Performance Computing (HPC) is the ability to process data and perform complex calculations at extremely high speeds. Current HPC platforms can achieve calculations on the order of quadrillions of calculations per second with quintillions on the horizon. The past three decades witnessed a vast increase in the use of HPC across different scientific, engineering and business communities, for example, sequencing the genome, predicting climate changes, designing modern aerodynamics, or establishing customer preferences. Although HPC has been well incorporated into science curricula such as bioinformatics, the same cannot be said for most computing programs. This working group will explore how HPC can make inroads into computer science education, from the undergraduate to postgraduate levels. The group will address research questions designed to investigate topics such as identifying and handling barriers that inhibit the adoption of HPC in educational environments, how to incorporate HPC into various curricula, and how HPC can be leveraged to enhance applied critical thinking and problem solving skills. Four deliverables include: (1) a catalog of core HPC educational concepts, (2) HPC curricula for contemporary computing needs, such as in artificial intelligence, cyberanalytics, data science and engineering, or internet of things, (3) possible infrastructures for implementing HPC coursework, and (4) HPC-related feedback to the CC2020 project.

Improving the feedback that novices receive from programming environments is an important and often overlooked aspect of computing education research. This work in progress examines the effects of various mechanisms by which environments deliver feedback to users. By providing insights on the effects of these mechanisms, we aim to inform designers, developers and educators about more effective design and use of such environments for students.

It is generally accepted that learning to program could be easier for many students. One of the most important components of this experience is the programming environment. Novices learn in a variety of environments, from basic command-line interfaces to industry-strength IDEs. These environments can differ substantially in compilation behavior and error message presentation - arguably two of the most important mechanisms through which users interact with the programming language. In this study, we utilize Blackbox data to compare the programming behavior of thousands of users programming in Java, who all used BlueJ versions 3 and 4. These two versions differ drastically in terms of compilation behavior and error message presentation. BlueJ 3 is a click-to-compile editor that delivers text-based error messages from javac to the user, but only presents the first error message, even if the compiler produces several. BlueJ 4 automatically compiles in the background but retains click-to-compile ability. In addition, all error messages (not just the first) may be viewed by the user. We find that the programming experience and behavior of these users can be substantially affected by changes in these mechanisms, causing numbers of manual compilations, successful compilations, and error messages presented in each version to differ, in cases, markedly. Our results provide evidence on how changes in programming environment affect user behavior in conditions that reasonably control for variables other than the programming environment. This can inform the decisions of educators, tool designers, and HCI researchers in their work to make learning more effective for novice programmers.

SIGCSE is a global organization with members from well over one hundred countries, but attendance at SIGCSE conferences is not always reflective of membership as a whole. Attendance at the SIGCSE Technical Symposium is overwhelmingly from the United States, with more than 92% of all attendees in recent years having a U.S. affiliation. This panel, which includes members of the SIGCSE Board and the Symposium International Committee, will present the state of Symposium participation from outside the U.S. in an effort towards understanding what can be done to expand global participation in the Symposium. The panelists span the range from students to long-time faculty and together have computing education experience from seven countries across six continents. The panel will also solicit views from attendees on barriers to global Symposium participation and ways to improve non-U.S. participation in SIGCSE's flagship conference.

Achieving high performance for facial age estimation with subjects in the borderline between adulthood and non-adulthood has always been a challenge. Several studies have used different approaches from the age of a baby to an elder adult and different datasets have been employed to measure the mean absolute error (MAE) ranging between 1.47 to 8 years. The weakness of the algorithms specifically in the borderline has been a motivation for this paper. In our approach, we have developed an ensemble technique that improves the accuracy of underage estimation in conjunction with our deep learning model (DS13K) that has been fine-tuned on the Deep Expectation (DEX) model. We have achieved an accuracy of 68% for the age group 16 to 17 years old, which is 4 times better than the DEX accuracy for such age range. We also present an evaluation of existing cloud-based and offline facial age prediction services, such as Amazon Rekognition, Microsoft Azure Cognitive Services, How-Old.net and DEX.

The goal of many computing education researchers is to improve how computing is taught and learned. To do that, researchers must engage with teachers, coaches, and mentors who design instructional materials and deliver lessons. However researchers may not be investigating problems that are directly of interest or utility to practitioners, and thus may not deliver results that are as impactful as possible in their contexts. To find out what research most interests today's practitioners, we conducted a two-stage survey. The first stage gathered questions that practitioners want researchers to investigate, and the second stage ranked these questions in terms of importance. We found that today's practitioners are more interested in student behavior, student understanding, and pedagogy than in languages and tools, curriculum, and inclusivity, and that there is little overlap between the questions ranked as highly interesting by researchers and those ranked highly by practitioners. Our results indicate that researchers need to better communicate why the questions they are pursuing are important, look for opportunities to collaborate with those who teach but have little direct connection with research, and examine the relevance of their research questions to practitioners.

The goal of most computing education research is to effect positive change in how computing is taught and learned. Statistical techniques are one important tool for achieving this goal. In this paper we report on an analysis of ICER papers that use inferential statistics. We present the most commonly used techniques; an overview of the techniques the ICER community has used over its first 14 years of papers, grouped according to the purpose of the technique; and a detailed analysis of three of the most commonly used techniques (t-test, chi-squared test, and Mann-Whitney-Wilcoxon). We identify common flaws in reporting and give examples of papers where statistics are reported well. In sum, the paper draws a picture of the use of inferential statistics by the ICER community. This picture is intended to help orient researchers who are new to the use of statistics in computing education research and to encourage reflection by the ICER community on how it uses statistics and how it can improve that use.

Between January and April of 2018, a comprehensive survey of introductory programming courses was undertaken across all sectors of Irish third-level institutions (universities, institutes of technology, and private colleges). The survey instrument was based on - and nearly identical to - recent surveys in the UK and Australasia. In total we report on 39 introductory programming courses at 25 third-level institutions. This includes 6 of 7 universities and 13 of 14 institutes of technology, representing 90% of all publicly funded institutions. We also report on 4 private colleges representing 80% of colleges in the Irish Higher Education Colleges Association that offer computing degrees.

Diagnostic messages generated by compilers and interpreters such as syntax error messages have been researched for decades. Unfortunately these messages which include error, warning, and runtime messages, present substantial difficulty and could be more effective, particularly for novices. Recent years have seen increased number of papers in the area including studies on the effectiveness of these messages, improving or enhancing them, and their usefulness as a part of programming process data that can be used to predict student performance. Despite this increased interest, the long history of literature is quite scattered and has not been brought together in any digestible form. We argue that in order to help the community proceed with more work on diagnostic messages, the literature needs to be presented in a state-of-the-art report. In addition we will synthesize and present the existing evidence for these messages including the difficulties they present and their effectiveness. We will also formulate a set of guidelines based on this evidence that can be used when designing or enhancing diagnostic messages. This work can serve as a starting point for those who wish to conduct research on such messages, those who wish to design better messages or those that aim to measure their effectiveness, more effectively.

We present an analysis of the visual portrayals of data (including results) in the full papers and working group reports of ITiCSE from 2013 to 2018. We find that tables are the most common visual portrayal of data in these publications, but that a number of graphical forms are also widely used. We examine the quality of the data portrayals for tables, graphs, and images using visual quality indicators derived from the literature. Overall, our findings are not positive. We find that many papers present data in such a way that it cannot be readily interpreted. The most common problem is captions that do not adequately describe the table, figure, or image. In tables, the main issues affecting readability of numeric data are poor alignment of numbers within a column, unnecessary notations, and unwarranted precision. In graphs and images, the prevalent problem is text that is too small to be read. We conclude with guidelines for future authors to ITiCSE and other computing education venues, in the hope that we can contribute to an improvement in the quality of computing education publications.

A well-received ITiCSE 2016 paper challenged the orthodox view that programming is hard to learn. It contended that CS1 educators' expectations are too high, which can result in poor teaching and learning, and could impact negatively on diversity and equity. The author posed a challenge to the community to collect research-based evidence of what novice programmers can achieve, and use evidence to derive realistic expectations for achievement. We argue that before rising to this challenge we must determine: What exactly do educators expect of introductory programming students? This paper presents our efforts toward answering this question. We manually curated hundreds of CS1 syllabi, providing a fresh perspective of expectation in CS1 courses. We analyzed learning outcomes and their concepts, in addition to languages utilized and other useful CS1 design and delivery information. This work contributes to a current picture of what is expected of introductory programming students, and provides an interactive online tool linked to all collected syllabi and containing all learning outcomes and other associated information. We hope this will aid the community in deciding whether or not we have unrealistic expectations of our CS1 students and if so, our contributions provide a starting point for the community to adjust them.

The SIGCSE Technical Symposium is celebrating its 50th year, and a constant theme throughout this history has been to better understand how novices learn to program. In this paper, we present a perspective on the evolution of introductory programming education research at the Symposium over these 50 years. We also situate the Symposium's impact in the context of the wider literature on introductory programming research. Applying a systematic approach to collecting papers presented at the Symposium that focus on novice programming / CS1, we categorized hundreds of papers according to their main focus, revealing important introductory programming topics and their trends from 1970 to 2018. Some of these topics have faded from prominence and are less relevant today while others, including many topics focused on students, such as making learning programming more appropriate from gender, diversity, accessibility and inclusion standpoints, have garnered significant attention more recently. We present discussions on these trends and in doing so, we provide a checkpoint for introductory programming research. This may provide insights for future research on how we teach novices and how they learn to program.

When solving programming problems, novices are often not aware of where they are in the problem-solving process. For instance, students who misinterpret the problem prompt will most likely not form a valid conceptual model of the task and fail to make progress towards a working solution. Avoiding such errors, and recovering from them once they occur, requires metacognitive skills that enable students to reflect on their problem-solving processes. For these reasons, developing metacognitive awareness is crucially important for novice students. Previous research has shown that explicitly teaching key steps of programming problem-solving, and having students reflect on where they are in the problem-solving process, can help students complete future programming assignments. Such metacognitive awareness training can be done through personal tutoring, but can be difficult to implement without a high ratio of instructors to students. We explore a more scalable approach, making use of an automated assessment tool, and conduct a controlled experiment to see whether scaffolding the problem-solving process would increase metacognitive awareness and improve student performance. We collected all code submissions by students in both control and experimental groups, as well as data from direct observation using a think-aloud protocol. We found that students who received the intervention showed a higher degree of understanding of the problem prompt and were more likely to complete the programming task successfully.

The problem of partitioning dense matrices into sets of sub-matrices has received increased attention recently and is crucial when considering dense linear algebra and kernels with similar communication patterns on heterogeneous platforms. The problem of load balancing and minimizing communication is traditionally reducible to an optimization problem that involves partitioning a square into rectangles. This problem has been proven to be NP-Complete for an arbitrary number of partitions. In this paper, we present recent approaches that relax the restriction that all partitions be rectangles. The first approach uses an original mathematical technique to find the exact optimal partitioning. Due to the complexity of the technique, it has been developed for a small number of partitions only. However, even at a small scale, the optimal partitions found by this approach are often non-rectangular and sometimes non-intuitive. The second approach is the study of approximate partitioning methods utilizing recursive partitioning algorithms. In particular we use the work on optimal partitioning to improve pre-existing algorithms. In this paper we discuss the different perspectives this approach opens and present two algorithms, SNRPP which is a$\sqrt{\frac{3}{2}}$approximation, and NRPP which is a$\frac{2}{\sqrt{3}}$approximation. While sub-optimal, the NRRP approach works for an arbitrary number of partitions. We use the first exact approach to analyse how close to the known optimal solutions the NRRP algorithm is for small numbers of partitions.

A broad review of research on the teaching and learning of programming was conducted by Robins et al. in 2003. Since this work there have been several reviews of research concerned with the teaching and learning of programming, in particular introductory programming. However, these reviews have focused on highly specific aspects, such as student misconceptions, teaching approaches, program comprehension, potentially seminal papers, research methods applied, automated feedback for exercises, competency-enhancing games, and program visualisation. While these aspects encompass a wide range of issues, they do not cover the full scope of research into novice programming. Some notable areas that have not been reviewed are assessment, academic integrity, and novice student attitudes to programming. There does not appear to have been a comprehensive review of research into introductory programming since that of Robins et al. It is therefore timely to conduct and present such a review in order to gain an understanding of the research focuses, to highlight advances in knowledge since 2003, and to indicate possible future directions for research. The working group will conduct a systematic literature review based on the guidelines proposed by Kitchenham et al. This research project is well suited to an ITiCSE working group as the synthesis and discussion of the literature will benefit from input from a variety of researchers drawn from different backgrounds and countries.

Conferences such as ITiCSE have recently seen an increase in the number of papers presenting empirical research in computing education. While empirical research need not be quantitative, there has been a corresponding increase in the number of papers that present some level of statistical analysis to support their arguments. This poster introduces a project that is exploring how -- and how well -- statistics are used and reported in computing education research.

There is an active strand of research in the literature exploring the effects of Enhanced Compiler Error Messages on student programming behavior, however many results seem conflicting. This is compounded by the fact that directly comparing these results is difficult as these studies utilize different metrics, and what metrics are best suited to measure the effects of enhanced compiler error messages is not known. Common to most studies to-date is that the metrics employed measure how many errors students produce, and/or rectify while writing programs. This study takes a different approach by measuring how many pre-existing syntax errors are rectified by students while debugging programs. Specifically, we measured the effect of enhanced compiler error messages in an empirical control/intervention experiment where students were given the task of removing syntax errors from non-compiling source code they did not write. We find a significant positive effect on the overall number of errors rectified, as well as the number of certain specific error types, but no significant effect on the number of non-compiling submissions or student scores. These results (in different ways) support the findings of several recent studies and suggest that their results may not be as conflicting as they seem. This is evidence that enhanced error messages may be effective, but also that the signal of these effects are relatively weak, indicating that how and what is measured when attempting to observe these effects is important.

In order to help students learning to develop computer programs, several computing education researchers have analyzed the compiler error messages generated by novices' attempts to compile their programs. The goal is to help students diagnose the errors they make through the messages generated by the compiler. This paper builds on that previous work by applying a technique based on a heuristic well-known to programmers - fix the first error and ignore the rest - to the analysis of over 21 million compiler error messages from the Blackbox dataset. We find that the ranks and frequencies obtained by considering all error messages are generally consistent with previously published lists, but when we consider first messages only, these ranks and frequencies are different. These differences could have important implications for teaching, and can inform tool design and future research efforts.

Replication research is rare in CS education. For this reason, it is often unclear to what extent our findings generalize beyond the context of their generation. The present paper is a replication and extension of Achievement Goal Theory research on CS1 students. Achievement goals are cognitive representations of desired competence (e.g., topic mastery, outperforming peers) in achievement settings, and can predict outcomes such as grades and interest. We study achievement goals and their effects on CS1 students at six institutions in four countries. Broad patterns are maintained --- mastery goals are beneficial while appearance goals are not --- but our data additionally admits fine-grained analyses that nuance these findings. In particular, students' motivations for goal pursuit can clarify relationships between performance goals and outcomes.

Current CS1 learning outcomes are relatively general, specifying tasks such as designing, implementing, testing and debugging programs that use some fundamental programming constructs. These outcomes impact what we teach, our expectations, and our assessments. Although prior work has demonstrated the utility of single concept assessments, most assessments used in formal examinations combine numerous heterogeneous concepts, resulting in complex and difficult tasks.

Encountering the same compiler error repeatedly, particularly several times consecutively, has been cited as a strong indicator that a student is struggling with important programming concepts. Despite this, there are relatively few studies which investigate repeated errors in isolation or in much depth. There are also few data-driven metrics for measuring programming performance, and fewer for measuring repeated errors. This paper makes two contributions. First we introduce a new metric to quantify repeated errors, the repeated error density (RED). We compare this to Jadud's Error Quotient (EQ), the most studied metric, and show that RED has advantages over EQ including being less context dependent, and being useful for short sessions. This allows us to answer two questions posited by Jadud in 2006 that have until now been unanswered. Second, we compare the EQ and RED scores using data from an empirical control/intervention group study involving an editor which enhances compiler error messages. This intervention group has been previously shown to have a reduced overall number of student errors, number of errors per student, and number of repeated student errors per compiler error message. In this research we find a reduction in EQ, providing further evidence that error message enhancement has positive effects. In addition we find a significant reduction in RED providing evidence that this metric is valid.

One of the many challenges novice programmers face from the time they write their first program is inadequate compiler error messages. These messages report details on errors the programmer has made and are the only feedback the programmer gets from the compiler. For students they play a particularly essential role as students often have little experience to draw upon, leaving compiler error messages as their primary guidance on error correction. However these messages are frequently inadequate, presenting a barrier to progress and are often a source of discouragement. We have designed and implemented an editor that provides enhanced compiler error messages and conducted a controlled empirical study with CS1 students learning Java. We find a reduced frequency of overall errors and errors per student. We also identify eight frequent compiler error messages for which enhancement has a statistically significant effect. Finally we find a reduced number of repeated errors. These findings indicate fewer students struggling with compiler error messages.

UNLABELLED: MicroRNAs are short non-coding RNA which function to fine-tune protein levels in all cells. This is achieved mainly by sequence-specific binding to 3' untranslated regions of target mRNA. The result is post-transcriptional interference in gene expression which reduces protein levels either by promoting destabilisation of mRNA or translational repression. Research published since 2010 shows that microRNAs are important regulators of gene expression in epilepsy. A series of microRNA profiling studies in rodent and human tissue has revealed that epilepsy is associated with wide ranging changes to microRNA levels in the brain. These are thought to influence processes including cell death, inflammation and re-wiring of neuronal networks. MicroRNAs have also been identified in the blood after injury to the brain and therefore may serve as biomarkers of epilepsy. EpimiRBase is a manually curated database for researchers interested in the role of microRNAs in epilepsy. The fully searchable database includes information on up- and down-regulated microRNAs in the brain and blood, as well as functional studies, and covers both rodent models and human epilepsy. AVAILABILITY AND IMPLEMENTATION: EpimiRBase is available at http://www.epimirbase.eu CONTACT: [email protected].

We present a new data partitioning strategy for parallel computing on three interconnected clusters. This partitioning has two advantages over existing partitionings. First it can reduce communication time due to a lower total volume of communication and a more efficient communication schedule. When the network topology is a linear array this partitioning always results in a lower total volume of communication compared to existing partitionings, provided the most powerful node is at the center of the array. When the topology is fully connected this partitioning results in a lower total volume of communication for all but a few power ratios. Second, it allows for the overlapping of communication and computation. These two inherent advantages work together to reduce overall execution time significantly.

This paper presents a new partitioning algorithm to perform matrix multiplication on two interconnected heterogeneous processors. Data is partitioned in a way which minimizes the total volume of communication between the processors compared to more general partitionings, resulting in a lower total execution time whenever the power ratio between the processors is greater than 3:1. The algorithm has interesting and important applicability, particularly as the top-level partitioning in a hierarchal algorithm that is to perform matrix multiplication on two interconnected clusters of computers