This research has three prongs, with each comparing open- and closed-book exam questions across six years (2017-2023) in a final year undergraduate applied machine learning course. First, the authors evaluated the performance of numerous LLMs, compared to student performance, and comparing open and closed book exams. Second, at a micro level, the examination questions and categories for which LLMs were most and least effective were compared. This level of analysis is rarely if ever, discussed in the literature. The research finally investigates LLM detection techniques, specifically their efficacy in identifying replies created wholly by an LLM. It considers both raw LLM outputs and LLM outputs that have been tampered with by students, with an emphasis on academic integrity. This study is a staff-student research collaboration, featuring contributions from eight academic professionals and six students.

Over the last year an online learning platform has been developed and piloted to the Irish second level education system allowing both students and teachers to participate in introductory computing modules. This poster will outline the development of the registration process of a system that is capable of managing potentially 728 schools, 1000+ classrooms and one million students (the entire Irish second level school system). CSLINC is an online student virtual learning environment for computing consisting of several modules built by academics and industry leaders and disseminated to schools through Moodle, our selected virtual learning environment. While Moodle has a certain amount of automation and user management built-in, this poster will present the initial design considerations and the automation process developed to allow for school centered mass registration on Moodle. This is of value to other CER educators who may consider developing such a system and enrollment process. Future work will consist of a detailed publication on the development process.

This work piloted an international replication and generalization study on an existing prediction model called PreSS. PreSS has been developed and validated over nearly two decades and can predict student performance in CS1 with nearly 71% accuracy, at a very early stage in the learning module. Motivated by a prior validation study and its competitive modelling accuracy, we chose PreSS for such an international replication and generalization study. The study took place in two countries, with two institutions in Ireland and one institution in the US, totalling 472 students throughout the academic year 2020-21. In doing so, this study addressed a call from the 2015 ITiCSE working group for the educational data mining and learning analytics community: systematically analyse and verify previous studies using data from multiple contexts to tease out tacit factors that contribute to previously observed outcomes. This pilot study achieved 90% accuracy, which is higher than the prior work's. This encouraging finding sets the foundations for a larger scale international study. This paper describes in detail the pilot replication and generalization study and our progress on the larger scale study which is taking place across six continents.

Creating Leaders in Coding Kishoge was a pilot intervention thatwas designed to try to encourage lower second level female studentsto continue studying Computer Science (CS). Research has shownthat increasing access alone to CS does not necessarily broaden participation for females [1]. Compounding this problem, a lack ofvisible role models in the field may contribute to female studentsbeing unable to envisage themselves in a CS role, or indeed understand the types of roles that are available. In the late 2000s, an EU action group funded an initiative, "Science: It's a girl's thing!" which further alienated females from those critical STEM roles. In an attempt to address these issues and change perceptions of CS at a young age, females from industry provided mentorship for the female students. Early findings would suggest that female students are deciding not to partake in CS at second level before entering second level education.

PreSS# is an online Machine Learning prediction model that aims to identify students at risk of failing or dropping out in an introductory programming course (typically called CS1). PreSS# has been developed over the past 16 years, where the model is capable of predicting at-risk students with an accuracy of 71%. There is, however, a need to re-validate the model using a larger international multi-jurisdictional multi-university data set, as up until now the data sets have been predominantly from a single jurisdiction. The goal of this study is to not only re-validate the model using a multi-jurisdictional data set, but, in line with a 2015 ITiCSE working group report's Grand Challenges, to openly publish the data set itself. This work timely to the CSEd community as other researchers can use this data to further their research, re-validate PreSS# and will be able to then contribute, by submitting their local PreSS# data sets to this global online repository.

Like many others, our institution had to adapt our traditional proctored, written examinations to open-book online variants due to theCOVID-19 pandemic. This paper describes the process applied to develop open-book online exams for final year (undergraduate)students studying Applied Machine Learning and Applied Artificial Intelligence and Deep Learning courses as part of a four-year BSc in Computer Science. We also present processes used to validate the examinations as well as plagiarism detection methods implemented. Findings from this study highlight positive effects of using open-book online exams, with ~85% of students reporting that they either prefer online open-book examinations or have no preference between traditional and open-book exams. There were no statistically significant differences reported comparing the exam results of student cohorts who took the open-book online examination, compared to previous cohorts who sat traditional exams. These results are of value to the CSEd community for three reasons. First, it outlines a methodology for developing online open-book exams(including publishing the open-book online exam papers as samples). Second, it provides approaches for deterring plagiarism and implementing plagiarism detection for open-book exams. Finally, we present feedback from students which may be used to guidefuture online open-book exam development.

The use of Education Data Mining (EDM) has seen a significant increase in recent years. A recent report identified notable concerns with the literature relating to the lack of metrics presented in EDM research (in particular, predicting student performance). This poster presents details on these concerns that may inhibit future re-validation studies or worse, models that initially report strong findings which may not generalise. This poster also declares a call to action for future studies to present such metrics, and finally describes ongoing work in this space (a systematic literature review).