This research has three prongs, with each comparing open- and closed-book exam questions across six years (2017-2023) in a final year undergraduate applied machine learning course. First, the authors evaluated the performance of numerous LLMs, compared to student performance, and comparing open and closed book exams. Second, at a micro level, the examination questions and categories for which LLMs were most and least effective were compared. This level of analysis is rarely if ever, discussed in the literature. The research finally investigates LLM detection techniques, specifically their efficacy in identifying replies created wholly by an LLM. It considers both raw LLM outputs and LLM outputs that have been tampered with by students, with an emphasis on academic integrity. This study is a staff-student research collaboration, featuring contributions from eight academic professionals and six students.

Over the last year an online learning platform has been developed and piloted to the Irish second level education system allowing both students and teachers to participate in introductory computing modules. This poster will outline the development of the registration process of a system that is capable of managing potentially 728 schools, 1000+ classrooms and one million students (the entire Irish second level school system). CSLINC is an online student virtual learning environment for computing consisting of several modules built by academics and industry leaders and disseminated to schools through Moodle, our selected virtual learning environment. While Moodle has a certain amount of automation and user management built-in, this poster will present the initial design considerations and the automation process developed to allow for school centered mass registration on Moodle. This is of value to other CER educators who may consider developing such a system and enrollment process. Future work will consist of a detailed publication on the development process.

Recent developments, guidelines, and acts by the European Commission have started to frame policy for AI and related areas such as ML and data, not only for the broader community, but in the context of education specifically. This poster presents a succinct overview of these developments. Specifically, we look to bring together all publications that might impact the teaching of AI (for example, teacher expectations in the coming years around AI competencies)and publications that affect the use of AI in the classroom. We mean using tools and systems that incorporate both 'Good Old Fashioned 'AI and those that can directly impact students. This poster is of value to both the European and the wider CER communities and practitioners, as it brings together several guidelines, acts, and plans that are not easily searchable or linked. The publications presented in this poster will impact the teaching of AI and teaching with Ai in Europe, and insights can be drawn and compared for other jurisdictions as the educational world adapts to and with AI.

The last ten years the world has experienced a revival in computing education with countries transiting to or adopting a compulsory computing education curriculum, which has created a resurgence in undergraduate students studying computing. Coinciding with this movement, the decolonization of curriculum began to collect momentum. This paper explores these two movements in education based on Freire’s Pedagogy of the Oppressed, and the influence on the design of computing curriculum, and instructional techniques for grades 9-16 computing modules. This review will inform computing curriculum designers and computing educators lessons learned in the application of social science learning theories in computing education curriculum design and delivery.

In 2021 CSINC (a TU Dublin research group) launched their online Computer Science MOOC called CSLINC(Computer Science Inclusive Learning Environment). CSLINC provides a suite of eight week courses in computer science (CS) which are scaffolded for teachers to support their delivery in the classroom Quille22. The uptake for CSLINC was significant with 230 teachers delivering material to 12,000 students. The goals of CSLINC are to attempt to change CS perceptions and increase the number of students choosing Leaving Certificate Computer Science (LCCS) and also to increase teacher programming self-efficacy in the delivery of CS materials, thus leading to more teachers considering to deliver the formal curricula. Previous research conducted in the area of teacher programming self-efficacy found that programming self-efficacy is lower for formal-curricula second-level CS teachers compared to CS1 Students Faherty21. In the study teachers were attending CPD aimed at those teaching or planning to teach the LCCS subject at upper second level. The purpose of this pilot study is to do an initial comparison between the programming self-efficacy of teachers who have used CSLINC (who do not deliver formal CS curricula) in the classroom compared to the teachers who are or were planning to deliver LCCS (formal curricula).

The increasing use of AI in industry and society not only expects but demands that we build human-centred competencies into our AI education programmes. The computing education community needs to adapt, and while the adoption of standalone ethics modules into AI programmes or the inclusion of ethical content into traditional applied AI modules is progressing, it is not enough. To foster student competencies to create AI innovations that respect and support the protection of individual rights and society, a novel ground-up approach is needed. This panel presents on one such approach, the development of a Human-Centred AI Masters (HCAIM) as well as the insights and lessons learned from the process. In particular, we discuss the design decisions that have led to the multi-institutional master's programme. Moreover, this panel allows for discussion on pedagogical and methodological approaches, content knowledge areas and the delivery of such a novel programme, along with challenges faced, to inform and learn from other educators that are considering developing such programmes.

This work piloted an international replication and generalization study on an existing prediction model called PreSS. PreSS has been developed and validated over nearly two decades and can predict student performance in CS1 with nearly 71% accuracy, at a very early stage in the learning module. Motivated by a prior validation study and its competitive modelling accuracy, we chose PreSS for such an international replication and generalization study. The study took place in two countries, with two institutions in Ireland and one institution in the US, totalling 472 students throughout the academic year 2020-21. In doing so, this study addressed a call from the 2015 ITiCSE working group for the educational data mining and learning analytics community: systematically analyse and verify previous studies using data from multiple contexts to tease out tacit factors that contribute to previously observed outcomes. This pilot study achieved 90% accuracy, which is higher than the prior work's. This encouraging finding sets the foundations for a larger scale international study. This paper describes in detail the pilot replication and generalization study and our progress on the larger scale study which is taking place across six continents.

We present an overview of a "Machine Learning and Artificial Intelligence" course that is part of a large online course platform for upper second level students. We take a novel approach to teaching fundamental AI concepts that does not require code, and assumes little prior knowledge including only basic mathematics. The design ethos is for students to gain an understanding of how algorithms can "learn". Many misconceptions exist about this term with respect to AI and can lead to confusion and more serious misconceptions, particularly for students who engage with AI-enabled tools regularly. This approach aims to provide insights into how AI actually works, to demystify and remove barriers to more advanced learning, and to emphasize the important roles of ethics and bias in AI. We took several steps to engage students, including videos narrated by a final-year second-level student (US 12th grade). We present design and logistics particulars on this course which is currently being taken by ~7,000 students in Ireland. We believe this will be of value to other educators and the wider community.

This poster introduces CSLINC, a free scaffolded MOOC framework tailored to second-level students in Ireland that consists of: an online platform built for accessibility; a suite of modules developed upon international best practices with varying co-creators; and automated assessment and certificates of completion. Its aim is to provide content to promote national CS curricula to all second-level students in Ireland. In September 2021, CSLINC launched to 10,000 students across 100 schools. Future work will include collecting and collating research to validate CSLINC's objectives, scaffolding that will build foundations for national curriculum learning outcomes, and measure its impact on students, their perceptions and follow on CS uptake at second-level in Ireland.

In 2018 Ireland launched a new, nationwide Computer Science curriculum for upper second-level students. The biggest challenge preventing uptake of the subject is a low supply of teachers qualified to teach the subject resulting in most schools not offering the subject, preventing most students from having the choice in the first place. Currently, approximately 150 secondary schools (of 723 nationally) offer the subject. Most school teachers who aspire to teach CS do not have a degree in computing and typically have very low self-efficacy in programming, as previously reported in the literature. There is a need to help these teachers to confidently de-liver the national CS curriculum at second-level in Ireland. CSLINC is a platform and MOOC offering teachers a scaffolded approach towards building their confidence for formal CS content delivery. CSLINC provides courses that teachers can deliver to junior second-level students before the official, national CS curriculum is offered at senior-level. This poster provides overview of the benefits of this system for second-level teachers in Ireland, which will also be of value to the computing education community internationally.

PreSS# is an online Machine Learning prediction model that aims to identify students at risk of failing or dropping out in an introductory programming course (typically called CS1). PreSS# has been developed over the past 16 years, where the model is capable of predicting at-risk students with an accuracy of 71%. There is, however, a need to re-validate the model using a larger international multi-jurisdictional multi-university data set, as up until now the data sets have been predominantly from a single jurisdiction. The goal of this study is to not only re-validate the model using a multi-jurisdictional data set, but, in line with a 2015 ITiCSE working group report's Grand Challenges, to openly publish the data set itself. This work timely to the CSEd community as other researchers can use this data to further their research, re-validate PreSS# and will be able to then contribute, by submitting their local PreSS# data sets to this global online repository.

Teacher self-esteem has been found to impact student learning in a number of non-computing fields. As computing slowly becomes a part of the upper secondary school (high school) curriculum in many countries, instruments designed to measure teachers' programming self-esteem can help inform classroom practice and processes such as teacher professional development needs. This study examines if there are differences in programming self-esteem (using the Bergin Programming Self-Esteem Instrument) between upper secondary school teachers and CS1 students in Ireland. In addition this study provides evidence of validity when using this instrument(originally developed for CS1 students) to measure upper secondary school teacher programming self-esteem. To test for evidence of validity, we compared the results of the programming self-esteem construct given to upper secondary school teachers (n=130) to a recent study of programming self-esteem among CS1 students (n=693). We found evidence of both reliability and validity with teachers that aligns with the evidence found for the CS1 students, demonstrating utility for use with teacher cohorts. Comparing these findings, teachers reported statistically significantly lower programming self-esteem compared to CS1 students. Interestingly CS1 students identifying as male had a statistically significant higher programming self-esteem than those identifying as female. However, we found no statistically significant difference for teacher gender, unlike previous work. Our results indicate that teacher programming self-esteem should be given consideration in the design and implementation of professional development.

Like many others, our institution had to adapt our traditional proctored, written examinations to open-book online variants due to theCOVID-19 pandemic. This paper describes the process applied to develop open-book online exams for final year (undergraduate)students studying Applied Machine Learning and Applied Artificial Intelligence and Deep Learning courses as part of a four-year BSc in Computer Science. We also present processes used to validate the examinations as well as plagiarism detection methods implemented. Findings from this study highlight positive effects of using open-book online exams, with ~85% of students reporting that they either prefer online open-book examinations or have no preference between traditional and open-book exams. There were no statistically significant differences reported comparing the exam results of student cohorts who took the open-book online examination, compared to previous cohorts who sat traditional exams. These results are of value to the CSEd community for three reasons. First, it outlines a methodology for developing online open-book exams(including publishing the open-book online exam papers as samples). Second, it provides approaches for deterring plagiarism and implementing plagiarism detection for open-book exams. Finally, we present feedback from students which may be used to guidefuture online open-book exam development.

The use of Education Data Mining (EDM) has seen a significant increase in recent years. A recent report identified notable concerns with the literature relating to the lack of metrics presented in EDM research (in particular, predicting student performance). This poster presents details on these concerns that may inhibit future re-validation studies or worse, models that initially report strong findings which may not generalise. This poster also declares a call to action for future studies to present such metrics, and finally describes ongoing work in this space (a systematic literature review).

This poster describes the use of online technology to deliver K-12 teacher professional development (PD) during the COVID-19 pandemic in Ireland. Traditionally these sessions are delivered in person, with a focus on hand-on activities, but the sudden changes faced by the closures in Ireland required an alternative approach for delivering these sessions. The PD session presented in this poster was a more technically challenging micro:bit workshop, which was delivered online using the micro:bit classroom. This is typically used as an in-class, one to many instructor tool, and trialing this as a PD collaborative tool, was a novel approach. This poster presents the delivery and methodology of the session, the collaborative online format, and feedback from the participants.

This paper outlines the longitudinal development of a K-12 outreach model, to promote Computer Science in Ireland. Over a three-year period, it has been piloted to just under 9700 K-12 students from almost every county in Ireland. The model consists of a two-hour camp that introduces students to a range of Computer Science topics: addressing computing perceptions, introduction to coding and exploration of computational thinking. The model incorporates on-site school delivery and is available at no cost to any interested school across Ireland. The pilot study so far collected over 3400 surveys (pre- and post- outreach delivery). Schools from all over Ireland self-selected to participate, including male only, female only and mixed schools. The no-cost nature of the model meant schools deemed "disadvantaged", to private fee-paying schools participated. Initial findings are very positive, including the balance of male and female participants, where in the 2017-18 academic year it was 56:44 and in 2019-20 (to date), it is 35:65 respectively. Once the model is validated and tweaked (based on survey data), the model will be published (open access) for other institutions to implement the model locally. In addition, the authors intend to link schools (that the team have worked with over the three years) with local institutions, thus developing a sustainable ecosystem for the program to continue. This paper describes the model structure and outlines early findings.

This paper describes a pilot intervention conducted in CS1, in the academic year of 2016-2017. The intervention was based on the work of Dweck, promoting a growth Mindset in an effort to increase performance in introductory programming. The study also examined data from a previous year (as a control group) to compare and contrast the results. Multiple factors related to programming performance were recorded with the control and treatment group, which were measured at multiple intervals throughout the course, to monitor changes as the pilot intervention was implemented. This study found a significant increase in programming performance when the intervention was deployed. However, although performance increased for the treatment group, the average Mindset did not significantly change towards a growth Mindset (replicating the findings of Cutts et al, 2010). To further explore this finding, a preliminary deeper investigation using k-means clustering was carried out. The investigation found that the intervention promoted a growth Mindset for some student profiles and a fixed Mindset for others. This finding is important for educators considering intervention development or implementation of Mindset, and demonstrates that a Mindset intervention may not be suitable for all learners.

Computer Science (CS) is a new subject area for many K-12 teachers around the world, requiring new disciplinary knowledge and skills. Teacher social-behavioral factors (e.g. self-esteem) have been found to impact learning and teaching, and a key part of CS curriculum implementation will need to ensure teachers feel confident to deliver CS. However, studies about CS teacher self-esteem are lacking. This paper presents an analysis of publicly available data (n=219) from a pilot study using a Teacher CS Self-Esteem scale. Analysis revealed significant differences, including 1) females reported significantly lower CS self-esteem than males, 2) primary teachers reported lower levels of CS self-esteem than secondary teachers, 3) those with no CS teaching experience reported significantly lower CS self-esteem, 4) teachers with 0-3 years experience had a negative CS self-esteem, but after four years, teachers had a positive CS self-esteem, and 5) teachers who lived further from metropolitan areas and in some countries reported lower CS self-esteem. These initial findings suggest a pressing need for future research to look further into teacher CS self-esteem to inform teacher CS professional development.

There has been a growing interest and increase in work shared about national K-12 Computer Science Education (CSED) curriculum and implementation efforts around the world. Much of this work focuses on curriculum analysis, country reports, experience reports and case studies. The K-12 CSED community would benefit from an international strategic effort to compare, contrast and monitor K-12 CSED over time, across multiple countries and regions, to understand pedagogy, practice, resources and experiences from the perspective of teachers working in classrooms. Furthermore, there is a need for validated and robust instruments that can support comparable investigations into the current state of K-12 CSED in schools around the world. Through a collaborative effort, this Working Group will develop a validated teacher survey instrument and collect data about CSED implementation and practice in K-12 classrooms. The authors will pilot the survey with K-12 teacher cohorts and analyse, compare and contrast survey findings across countries.

The SIGCSE Technical Symposium is celebrating its 50th year, and a constant theme throughout this history has been to better understand how novices learn to program. In this paper, we present a perspective on the evolution of introductory programming education research at the Symposium over these 50 years. We also situate the Symposium's impact in the context of the wider literature on introductory programming research. Applying a systematic approach to collecting papers presented at the Symposium that focus on novice programming / CS1, we categorized hundreds of papers according to their main focus, revealing important introductory programming topics and their trends from 1970 to 2018. Some of these topics have faded from prominence and are less relevant today while others, including many topics focused on students, such as making learning programming more appropriate from gender, diversity, accessibility and inclusion standpoints, have garnered significant attention more recently. We present discussions on these trends and in doing so, we provide a checkpoint for introductory programming research. This may provide insights for future research on how we teach novices and how they learn to program.

This paper describes a large, multi-institutional revalidation study conducted in the academic year 2015-16. Six hundred and ninety-two students participated in this study, from 11 institutions (ten institutions in Ireland and one in Denmark). The primary goal was to validate and further develop an existing computational prediction model called Predict Student Success (PreSS). In doing so, this study addressed a call from the 2015 ITiCSE working group (the second ”Grand Challenge”), to ”systematically analyse and verify previous studies using data from multiple contexts to tease out tacit factors that contribute to previously observed outcomes”.

This paper describes a multivariate, multi-institutional study conducted in the academic year 2015-16. Six hundred and ninety-three students participated from 11 institutions, (ten institutions in Ireland and one in Denmark). The goal of the study was to compare the profile of male and female students enrolled on introductory programming modules (CS1), to determine if any significant differences could be identified by gender. The gender split was 79:21, male to female respectively. The study took place early in the CS1 module with three instruments used to capture data: a background survey, a survey on programming self-efficacy, comfort and anxiety, and a short programming test. At the end of the module, the overall result for each participant was gathered. Of importance, the study was conducted across multiple levels of Computer Science education, from Level 5 Certificate up to and including Honors Bachelor Degree and Higher Diploma, (which are based on the Irish National Framework of Qualifications NFQ). This paper describes the approach taken and the detailed analysis performed. Several significant differences between male and female students were identified early in CS1, some of which did not hold true at the end of the module. A gender comparison between the two participating countries and the different institution types was also performed and discussed. The findings could be used to positively influence teaching practice and to the development of gender focused retention and recruitment strategies.