The expansion of large-scale online education platforms has yielded vast amounts of student interaction data for knowledge tracing (KT). KT models estimate students’ concept mastery from interaction data, but the models’ performance is sensitive to input data quality. Gaming behaviors, such as excessive hint use, may misrepresent students’ knowledge and undermine model reliability. However, systematic investigations of how different types of gaming behaviors affect KT remain scarce, and existing studies rely on costly manual analysis that does not capture behavioral diversity. In this study, we conceptualize gaming behaviors as a form of data poisoning, defined as the deliberate submission of incorrect or misleading interaction data to corrupt a model’s learning process. We design Data Poisoning Attacks (DPA) to simulate diverse gaming patterns and systematically evaluate their impact on KT model performance. Moreover, drawing on advances in DPA detection, we explore unsupervised approaches to enhance the generalizability of gaming behavior detection. We find that KT models performance tend to decrease especially for random guess behaviors. Our findings provide insights into the vulnerabilities of KT models and highlight the potential of adversarial methods for improving the robustness of learning analytics systems.

The learning analytics (LA) community has recently reached two important milestones: celebrating the 15th LAK conference and updating the 2011 definition of LA to reflect the 15 years of changes in the discipline. However, despite LA’s growth, little is known about how research topics, funding, and collaboration, as well as the relationships among them, have developed within the community over time. This study addressed this gap by analyzing all 936 full and short papers published at LAK over a 15-year period using unsupervised machine learning, natural language processing, and network analytics. The analysis revealed a stable core of prolific authors alongside high turnover of newcomers, systematic links between funding sources and research directions, and six enduring topical centers that remain globally shared but vary in prominence across countries. These six topical centers, which encompass LA research, are: self-regulated learning, dashboards and theory, social learning, automated feedback, multimodal analytics, and outcome prediction. Our findings highlight key challenges for the future: widening participation, reducing dependency on a narrow set of funders, and ensuring that emerging research trajectories remain responsive to educational practice and societal needs.

Background and context : Open datasets play a crucial role in three prominent research domains that intersect data science and education: learning analytics, educational data mining, and AI in education. Researchers in these domains apply computational methods to analyze data from educational contexts, aiming to better understand and improve teaching and learning. Research scope and gap : Providing open datasets alongside research papers supports research reproducibility, fosters collaboration, and increases trust in research findings. It also provides individual benefits for authors, such as greater visibility, credibility, and citation potential. However, despite these advantages, the availability of open datasets and the associated practices within the learning analytics research communities, especially at their flagship conference venues, remain unclear. Goal and method : To address this gap, we conducted a systematic survey of publicly available datasets published alongside research papers in learning analytics domains. We manually examined 1,125 papers from three respected flagship conferences (LAK, EDM, and AIED) over the past five years (2020–2024). We discovered, categorized, and analyzed 172 unique datasets used in 204 publications. Results and contributions : Our study presents the most comprehensive collection and analysis of open educational datasets to date, along with the most detailed categorization. Of the 172 datasets identified, 143 were not captured in any prior survey of open data in learning analytics. We provide insights into the datasets’ context, analytical methods, use, and other properties. Based on this survey, we summarize the current gaps in the field. Furthermore, we list practical recommendations, advice, and 8-item guidelines under the acronym PRACTICE with a checklist to help researchers publish their data. Lastly, we share our original dataset: an annotated inventory detailing the discovered datasets and the corresponding publications. We hope these findings will support further adoption of open data practices in learning analytics communities and beyond.

In supervised machine learning (SML) research, large training datasets are essential for valid results. However, obtaining primary data in learning analytics (LA) is challenging. Data augmentation can address this by expanding and diversifying data, though its use in LA remains underexplored. This paper systematically compares data augmentation techniques and their impact on prediction performance in a typical LA task: prediction of academic outcomes. Augmentation is demonstrated on four SML models, which we successfully replicated from a previous LAK study based on AUC values. Among 21 augmentation techniques, SMOTE-ENN sampling performed the best, improving the average AUC by 0.01 and approximately halving the training time compared to the baseline models. In addition, we compared 99 combinations of chaining 21 techniques, and found minor, although statistically significant, improvements across models when adding noise to SMOTE-ENN (+0.014). Notably, some augmentation techniques significantly lowered predictive performance or increased performance fluctuation related to random chance. This paper's contribution is twofold. Primarily, our empirical findings show that sampling techniques provide the most statistically reliable performance improvements for LA applications of SML, and are computationally more efficient than deep generation methods with complex hyperparameter settings. Second, the LA community may benefit from validating a recent study through independent replication.

Improving cybersecurity education has become a priority for many countries and organizations worldwide. Computing societies and professional associations have recognized cybersecurity as a distinctive computing discipline and created specialized cybersecurity curricular guidelines. Higher education institutions are introducing new cybersecurity programs, attracting students to this expanding field. In this paper, we examined 101 study programs across 24 countries. Based on their analysis, we argue that top-ranked universities have not yet fully implemented the guidelines and offer programs that have "cyber" in their name but lack some essential elements of a cybersecurity program. In particular, most programs do not sufficiently cover non-technical components, such as law, policies, or risk management. Also, most programs teach knowledge and skills but do not expose students to experiential learning outside the traditional classroom (such as internships) to develop their competencies. As a result, graduates of these programs may not meet employer expectations and may require additional training. To help program directors and educators improve their programs and courses, this paper offers examples of effective practices from cybersecurity programs around the world and our teaching practice.

This full paper in the research track evaluates the usage of data logged from cybersecurity exercises in order to predict students who are potentially at risk of performing poorly. Hands-on exercises are essential for learning since they enable students to practice their skills. In cybersecurity, hands-on exercises are often complex and require knowledge of many topics. Therefore, students may miss solutions due to gaps in their knowledge and become frustrated, which impedes their learning. Targeted aid by the instructor helps, but since the instructor's time is limited, efficient ways to detect struggling students are needed. This paper develops automated tools to predict when a student is having difficulty. We formed a dataset with the actions of 313 students from two countries and two learning environments: KYPO CRP and EDURange. These data are used in machine learning algorithms to predict the success of students in exercises deployed in these environments. After extracting features from the data, we trained and cross-validated eight classifiers for predicting the exercise outcome and evaluated their predictive power. The contribution of this paper is comparing two approaches to feature engineering, modeling, and classification performance on data from two learning environments. Using the features from either learning environment, we were able to detect and distinguish between successful and struggling students. A decision tree classifier achieved the highest balanced accuracy and sensitivity with data from both learning environments. The results show that activity data from cybersecurity exercises are suitable for predicting student success. In a potential application, such models can aid instructors in detecting struggling students and providing targeted help. We publish data and code for building these models so that others can adopt or adapt them.

Programming courses can be challenging for first year university students, especially for those without prior coding experience.Students initially struggle with code syntax, but as more advanced topics are introduced across a semester, the difficulty in learning to program shifts to learning computational thinking (e.g., debugging strategies).This study examined the relationships between students' rate of programming errors and their grades on two exams.Using an online integrated development environment, data were collected from 280 students in a Java programming course.The course had two parts.The first focused on introductory procedural programming and culminated with exam 1, while the second part covered more complex topics and object-oriented programming and ended with exam 2. To measure students' programming abilities, 51095 code snapshots were collected from students while they completed assignments that were autograded based on unit tests.Compiler and runtime errors were extracted from the snapshots, and three measures -Error Count, Error Quotient and Repeated Error Density -were explored to identify the best measure explaining variability in exam grades.Models utilizing Error Quotient outperformed the models using the other two measures, in terms of the explained variability in grades and Bayesian Information Criterion.Compiler errors were significant predictors of exam 1 grades but not exam 2 grades; only runtime errors significantly predicted exam 2 grades.The findings indicate that leveraging Error Quotient with multiple error types (compiler and runtime) may be a better measure of students' introductory programming abilities, though still not explaining most of the observed variability.

For undergraduate students of computing, learning to solve complex practical problems in a team is an essential skill for their future careers. This skill is needed in various fields, such as in cybersecurity and IT governance. Tabletop exercises are an innovative teaching method used in practice for training teams in incident response and evaluation of contingency plans. However, tabletop exercises are not yet widely established in university education. This paper presents data and teaching experience from a cybersecurity course that introduces tabletop exercises in classrooms using a novel technology: INJECT Exercise Platform (IXP), a web-based learning environment for delivering and evaluating the exercises. This technology substantially improves the prior practice, since tabletop exercises worldwide have usually been conducted using pen and paper. Unlike in traditional tabletop exercises, which are difficult to evaluate manually, IXP provides insights into students' behavior and learning based on automated analysis of interaction data. We demonstrate IXP's capabilities and evolution by comparing exercise sessions hosted throughout three years at different stages of the platform's readiness. The analysis of student data is supplemented by the discussion of the lessons learned from employing IXP in computing education contexts. The data analytics enabled a detailed comparison of the teams' performance and behavior. Instructors who consider innovating their classes with tabletop exercises may use IXP and benefit from the insights in this paper.

Tabletop exercises are used to train personnel in the efficient mitigation and resolution of incidents. They are applied in practice to support the preparedness of organizations and to highlight inefficient processes. Since tabletop exercises train competencies required in the workplace, they have been introduced into computing courses at universities as an innovation, especially within cybersecurity curricula. To help computing educators adopt this innovative method, we survey academic publications that deal with tabletop exercises. From 140 papers we identified and examined, we selected 14 papers for a detailed review. The results show that the existing research deals predominantly with exercises that follow a linear format and exercises that do not systematically collect data about trainees' learning. Computing education researchers can investigate novel approaches to instruction and assessment in the context of tabletop exercises to maximize the impact of this teaching method. Due to the relatively low number of published papers, the potential for future research is immense. Our review provides researchers, tool developers, and educators with an orientation in the area, a synthesis of trends, and implications for further work.

As cyber threats endanger everyone, from regular users to computing professionals, spreading cybersecurity awareness becomes increasingly critical. Therefore, our university designed an innovative cybersecurity awareness course that is freely available online for students, employees, and the general public. The course offers simple, actionable steps that anyone can use to implement defensive countermeasures. Compared to other resources, the course not only suggests learners what to do, but explains why and how to do it. To measure the course impact, we administered it to 138 computer science undergraduates within a compulsory information security and cryptography course. They completed the course as a part of their homework and filled out a questionnaire after each lesson. Analysis of the questionnaire responses revealed that the students valued the course highly. They reported new learning, perspective changes, and transfer to practice. Moreover, they suggested suitable improvements to the course. Based on the results, we have distilled specific insights to help security educators design similar courses. Lessons learned from this study are relevant for cybersecurity instructors, course designers, and educational managers.

Cybersecurity students need to develop practical skills such as using command-line tools. Hands-on exercises are the most direct way to assess these skills, but assessing students' mastery is a challenging task for instructors. We aim to alleviate this issue by modeling and visualizing student progress automatically throughout the exercise. The progress is summarized by graph models based on the shell commands students typed to achieve discrete tasks within the exercise. We implemented two types of models and compared them using data from 46 students at two universities. To evaluate our models, we surveyed 22 experienced computing instructors and qualitatively analyzed their responses. The majority of instructors interpreted the graph models effectively and identified strengths, weaknesses, and assessment use cases for each model. Based on the evaluation, we provide recommendations to instructors and explain how our graph models innovate teaching and promote further research. The impact of this paper is threefold. First, it demonstrates how multiple institutions can collaborate to share approaches to modeling student progress in hands-on exercises. Second, our modeling techniques generalize to data from different environments to support student assessment, even outside the cybersecurity domain. Third, we share the acquired data and open-source software so that others can use the models in their classes or research.

Networking, operating systems, and cybersecurity skills are exercised best in an authentic environment. Students work with real systems and tools in a lab environment and complete assigned tasks. Since all students typically receive the same assignment, they can consult their approach and progress with an instructor, a tutoring system, or their peers. They may also search for information on the Internet. Having the same assignment for all students in class is standard practice efficient for learning and developing skills. However, it is prone to cheating when used in a summative assessment such as graded homework, a mid-term test, or a final exam. Students can easily share and submit correct answers without completing the assignment. In this paper, we discuss methods for automatic problem generation for hands-on tasks completed in a computer lab environment. Using this approach, each student receives personalized tasks. We developed software for generating and submitting these personalized tasks and conducted a case study. The software was used for creating and grading a homework assignment in an introductory security course enrolled by 207 students. The software revealed seven cases of suspicious submissions, which may constitute cheating. In addition, students and instructors welcomed the personalized assignments. Instructors commented that this approach scales well for large classes. Students rarely encountered issues while running their personalized lab environment. Finally, we have released the open-source software to enable other educators to use it in their courses and learning environments.

This Research To Practice Full Paper presents how learning experience influences students' capability to learn and their motivation for further learning. Although each student is different, standard instruction methods do not adapt to individual students. Adaptive learning reverses this practice and attempts to improve the student experience. While adaptive learning is well-established in programming, it is rarely used in cybersecurity education. This paper is one of the first works investigating adaptive learning in cybersecurity training. First, we analyze the performance of 95 students in 12 training sessions to understand the limitations of the current training practice. Less than half of the students (45 out of 95) completed the training without displaying any solution, and only in two sessions, all students completed all phases. Then, we simulate how students would proceed in one of the past training sessions if it would offer more paths of various difficulty. Based on this simulation, we propose a novel tutor model for adaptive training, which considers students' proficiency before and during an ongoing training session. The proficiency is assessed using a pre-training questionnaire and various in-training metrics. Finally, we conduct a case study with 24 students and new training using the proposed tutor model and adaptive training format. The results show that the adaptive training does not overwhelm students as the original static training format. In particular, adaptive training enables students to enter several alternative training phases with lower difficulty than the phases in the original training. The proposed adaptive format is not restricted to particular training used in our case study. Therefore, it can be applied to practicing any cybersecurity topic or even in other related computing fields, such as networking or operating systems. Our study indicates that adaptive learning is a promising approach for improving the student experience in cybersecurity education. We also highlight diverse implications for educational practice that improve students' experience.

This Full Paper in the Innovative Practice category presents and evaluates a technical innovation for hands-on classes. When learning cybersecurity, operating systems, or networking, students perform practical tasks using a broad range of command-line tools. Collecting and analyzing data about the command usage can reveal valuable insights into how students progress and where they make mistakes. However, few learning environments support recording and inspecting command-line inputs, and setting up an efficient infrastructure for this purpose is challenging. To aid engineering and computing educators, we share the design and implementation of an open-source toolset for logging commands that students execute on Linux machines. Compared to basic solutions, such as shell history files, the toolset's novelty and added value are threefold. First, its configuration is automated so that it can be easily used in classes on different topics. Second, it collects metadata about the command execution, such as a timestamp, hostname, and IP address. Third, all data are instantly forwarded to central storage in a unified, semi-structured format. This enables automated processing of the data, both in real-time and post hoc, to enhance the instructors' understanding of student actions. The toolset works independently of the teaching content, the training network's topology, or the number of students working in parallel. We demonstrated the toolset's value in two learning environments at four training sessions. Over two semesters, 50 students played educational cybersecurity games using a Linux command-line interface. Each training session lasted approximately two hours, during which we recorded 4439 shell commands. The semiautomated data analysis revealed different solution patterns, used tools, and misconceptions of students. Our insights from creating the toolset and applying it in teaching practice are relevant for instructors, researchers, and developers of learning environments. We provide the software and data resulting from this work so that others can use them in their hands-on classes.

This Innovative Practice full paper describes a technical innovation for scalable teaching of cybersecurity hands-on classes using interactive learning environments. Hands-on experience significantly improves the practical skills of learners. However, the preparation and delivery of hands-on classes usually do not scale. Teaching even small groups of students requires a substantial effort to prepare the class environment and practical assignments. Further issues are associated with teaching large classes, providing feedback, and analyzing learning gains. We present our research effort and practical experience in designing and using learning environments that scale up hands-on cybersecurity classes. The environments support virtual networks with full-fledged operating systems and devices that emulate realworld systems. The classes are organized as simultaneous training sessions with cybersecurity assignments and learners' assessment. For big classes, with the goal of developing learners' skills and providing formative assessment, we run the environment locally, either in a computer lab or at learners' own desktops or laptops. For classes that exercise the developed skills and feature summative assessment, we use an on-premises cloud environment. Our approach is unique in supporting both types of deployment. The environment is described as code using open and standard formats, defining individual hosts and their networking, configuration of the hosts, and tasks that the students have to solve. The environment can be repeatedly created for different classes on a massive scale or for each student on-demand. Moreover, the approach enables learning analytics and educational data mining of learners' interactions with the environment. These analyses inform the instructor about the student's progress during the class and enable the learner to reflect on a finished training. Thanks to this, we can improve the student class experience and motivation for further learning. Using the presented environments KYPO Cyber Range Platform and Cyber Sandbox Creator, we delivered the classes on-site or remotely for various target groups of learners (K-12, university students, and professional learners). The learners value the realistic nature of the environments that enable exercising theoretical concepts and tools. The instructors value time-efficiency when preparing and deploying the hands-on activities. Engineering and computing educators can freely use our software, which we have released under an open-source license. We also provide detailed documentation and exemplary hands-on training to help other educators adopt our teaching innovations and enable sharing of reusable components within the community.

Cybersecurity serious games provide hands-on training of cybersecurity skills and enhance security awareness. Besides the learning content, they use gamification elements to engage and motivate the players. We propose guidelines for creating technical cybersecurity games in a higher education context, based on a~literature review and experience of cybersecurity instructors. We also introduce topics for further research in this area.

The Stack is a free open repository of learning activities for adults that illustrate computing principles without a computer. We explain the rationale behind its development, describe its content with an example, and discuss its applications in university teaching practice.

Hands-on training is an effective way to practice theoretical cybersecurity concepts and increase participants' skills. In this article, we discuss the application of visual analytics principles to the design, execution, and evaluation of training sessions. We propose a conceptual model employing visual analytics that supports the sensemaking activities of users involved in various phases of the training life cycle. The model emerged from our long-term experience in designing and organizing diverse hands-on cybersecurity training sessions. It provides a classification of visualizations and can be used as a framework for developing novel visualization tools supporting phases of the training life-cycle. We demonstrate the model application on examples covering two types of cybersecurity training programs.

This Full Paper in the Research-To-Practice Category presents a long-term study about the effects of a student-centered course on communication and professional skills on students' thoughts, attitudes, and behavior. The course is offered at a European university as part of a computer science master's program. This paper shares the design and challenges of a longitudinal study that reaches ten years behind and employs a mixed-methods approach. Besides presenting and interpreting the findings, we shed light on which features tend to stay on students' minds and impact their way of being and acting in society. Moreover, we suggest implications for the design and practice in comparable courses to maximize constructive, sustainable effects, such as improved active listening, presentation skills, and openness to other perspectives. These are essential (not only) for computer science professionals. Our findings suggest that the course provided significant learning for the vast majority of respondents, including aspects such as presenting while keeping the other side in mind, managing one's stress, and becoming less shy to speak up. All in all, we aim to contribute an evidence-based source of motivation for instructors in technically focused curricula who hold a student-centered stance.

Student teaching assistants are not uncommon in computer science. However, their pedagogical training is often only superficial. This poster presents the Teaching Lab - a mature and fully developed training course for novice teachers (mostly undergraduate teaching assistants), its core principles, content and unique features as it evolved over five years. Our experience can be helpful to others intending to create or adjust a training program for novice teachers.

There are different requirements on cybersecurity of industrial control systems and information technology systems. This fact exacerbates the global issue of hiring cybersecurity employees with relevant skills. In this paper, we present KYPO4INDUSTRY training facility and a course syllabus for beginner and intermediate computer science students to learn cybersecurity in a simulated industrial environment. The training facility is built using open-source hardware and software and provides reconfigurable modules of industrial control systems. The course uses a flipped classroom format with hands-on projects: the students create educational games that replicate real cyber attacks. Throughout the semester, they learn to understand the risks and gain capabilities to respond to cyber attacks that target industrial control systems. Our described experience from the design of the testbed and its usage can help any educator interested in teaching cybersecurity of cyber-physical systems.

Cybersecurity is now more important than ever, and so is education in this field. However, the cybersecurity domain encompasses an extensive set of concepts, which can be taught in different ways and contexts. To understand the state of the art of cybersecurity education and related research, we examine papers from the ACM SIGCSE and ACM ITiCSE conferences. From 2010 to 2019, a total of 1,748 papers were published at these conferences, and 71 of them focus on cybersecurity education. The papers discuss courses, tools, exercises, and teaching approaches. For each paper, we map the covered topics, teaching context, evaluation methods, impact, and the community of authors. We discovered that the technical topic areas are evenly covered (the most prominent being secure programming, network security, and offensive security), and human aspects, such as privacy and social engineering, are present as well. The interventions described in SIGCSE and ITiCSE papers predominantly focus on tertiary education in the USA. The subsequent evaluation mostly consists of collecting students' subjective perceptions via questionnaires. However, less than a third of the papers provide supplementary materials for other educators, and none of the authors published their dataset. Our results provide orientation in the area, a synthesis of trends, and implications for further research. Therefore, they are relevant for instructors, researchers, and anyone new in the field of cybersecurity education. The information we collected and synthesized from individual papers are organized in a publicly available dataset.

The concept of Capture the Flag (CTF) games for practicing cybersecurity skills is widespread in informal educational settings and leisure-time competitions. However, it is not much used in university courses. This paper summarizes our experience from using jeopardy CTF games as homework assignments in an introductory undergraduate course. Our analysis of data describing students' in-game actions and course performance revealed four aspects that should be addressed in the design of CTF tasks: scoring, scaffolding, plagiarism, and learning analytics capabilities of the used CTF platform. The paper addresses these aspects by sharing our recommendations. We believe that these recommendations are useful for cybersecurity instructors who consider using CTF games for assessment in university courses and developers of CTF game frameworks.

Capture the Flag games are software applications designed to exercise cybersecurity concepts, practice using security tools, and understand cyber attacks and defense. We develop and employ these games at our university for training purposes, unlike in the traditional competitive setting. During the gameplay, it is possible to collect data about players' in-game actions, such as typed commands or solution attempts, including the timing of these actions. Although such data was previously employed in computer security research, to the best of our knowledge, there were few attempts to use this data primarily to improve education. In particular, we see an open and challenging research problem in creating an artificial intelligence assistant that would facilitate the learning of each player. Our goal is to propose, apply, and experimentally evaluate data analysis and machine learning techniques to derive information about the players' interactions from the in-game data. We want to use this information to automatically provide each player with a personalized formative assessment. Such assessment will help the players identify their mastered concepts and areas for improvement, along with suggestions and actionable steps to take. Furthermore, we want to identify high- or low-performing players during the game, and subsequently, offer them game tasks more suitable to their skill level. These interventions would supplement or even replace feedback from instructors, which would significantly increase the learning impact of the games, enable more students to learn cybersecurity skills at an individual pace, and lower the costs.

Capture the Flag games are software applications designed to exercise cybersecurity concepts, practice using security tools, and understand cyber attacks and defense. We develop and employ these games at our university for training purposes, unlike in the traditional competitive setting. During the gameplay, it is possible to collect data about players' in-game actions, such as typed commands or solution attempts, including the timing of these actions. Although such data was previously employed in computer security research, to the best of our knowledge, there were few attempts to use this data primarily to improve education. In particular, we see an open and challenging research problem in creating an artificial intelligence assistant that would facilitate the learning of each player. Our goal is to propose, apply, and experimentally evaluate data analysis and machine learning techniques to derive information about the players' interactions from the in-game data. We want to use this information to automatically provide each player with a personalized formative assessment. Such assessment will help the players identify their mastered concepts and areas for improvement, along with suggestions and actionable steps to take. Furthermore, we want to identify high- or low-performing players during the game, and subsequently, offer them game tasks more suitable to their skill level. These interventions would supplement or even replace feedback from instructors, which would significantly increase the learning impact of the games, enable more students to learn cybersecurity skills at an individual pace, and lower the costs.

Many starting teachers of computer science have great professional skill but often lack pedagogical training. Since providing expert mentorship directly during their lessons would be quite costly, institutions usually offer separate teacher training sessions for novice instructors. However, the reflection on teaching performed with a significant delay after the taught lesson limits the possible impact on teachers. To bridge this gap, we introduced a weekly semi-structured reflective practice to supplement the teacher training sessions at our faculty. We created a paper diary that guides the starting teachers through the process of reflection. Over the course of the semester, the diary poses questions of increasing complexity while also functioning as a reference to the topics covered in teacher training. Piloting the diary on a group of 25 novice teaching assistants resulted in overwhelmingly positive responses and provided the teacher training sessions with valuable input for discussion. The diary also turned out to be applicable in a broader context: it was appreciated and used by several experienced university teachers from multiple faculties and even some high-school teachers. The diary is freely available online, including source and print versions.

Capture the Flag games represent a popular method of cybersecurity training. Providing meaningful insight into the training progress is essential for increasing learning impact and supporting participants' motivation, especially in advanced hands-on courses. In this paper, we investigate how to provide valuable post-game feedback to players of serious cybersecurity games through interactive visualizations. In collaboration with domain experts, we formulated user requirements that cover three cognitive perspectives: gameplay overview, person-centric view, and comparative feedback. Based on these requirements, we designed two interactive visualizations that provide complementary views on game results. They combine a known clustering and time-based visual approaches to show game results in a way that is easy to decode for players. The purposefulness of our visual feedback was evaluated in a usability field study with attendees of the Summer School in Cyber Security. The evaluation confirmed the adequacy of the two visualizations for instant post-game feedback. Despite our initial expectations, there was no strong preference for neither of the visualizations in solving different tasks.

This Work-In-Progress Paper for the Innovative Practice Category presents a novel experiment in active learning of cybersecurity. We introduced a new workshop on hacking for an existing science-popularizing program at our university. The workshop participants, 28 teenagers, played a cybersecurity game designed for training undergraduates and professionals in penetration testing. Unlike in learning environments that are simplified for young learners, the game features a realistic virtual network infrastructure. This allows exploring security tools in an authentic scenario, which is complemented by a background story. Our research aim is to examine how young players approach using cybersecurity tools by interacting with the professional game. A preliminary analysis of the game session showed several challenges that the workshop participants faced. Nevertheless, they reported learning about security tools and exploits, and 61% of them reported wanting to learn more about cybersecurity after the workshop. Our results support the notion that young learners should be allowed more hands-on experience with security topics, both in formal education and informal extracurricular events.

Adversary thinking is an essential skill for cybersecurity experts, enabling them to understand cyber attacks and set up effective defenses. While this skill is commonly exercised by Capture the Flag games and hands-on activities, we complement these approaches with a key innovation: undergraduate students learn methods of network attack and defense by creating educational games in a cyber range. In this paper, we present the design of two courses, instruction and assessment techniques, as well as our observations over the last three semesters. The students report they had a unique opportunity to deeply understand the topic and practice their soft skills, as they presented their results at a faculty open day event. Their peers, who played the created games, rated the quality and educational value of the games overwhelmingly positively. Moreover, the open day raised awareness about cybersecurity and research and development in this field at our faculty. We believe that sharing our teaching experience will be valuable for instructors planning to introduce active learning of cybersecurity and adversary thinking.

Cybersecurity games are an attractive and popular method of active learning. However, the majority of current games are created for advanced players, which often leads to frustration in less experienced learners. Therefore, we decided to focus on a diagnostic assessment of participants entering the games. We assume that information about the players' knowledge, skills, and experience enables tutors or learning environments to suitably assist participants with game challenges and maximize learning in their virtual adventure. In this paper, we present a pioneering experiment examining the predictive value of a short quiz and self-assessment for identifying learners' readiness before playing a cybersecurity game. We hypothesized that these predictors would model players' performance. A linear regression analysis showed that the game performance can be accurately predicted by well-designed prerequisite testing, but not by self-assessment. At the same time, we identified major challenges related to the design of pretests for cybersecurity games: calibrating test questions with respect to the skills relevant for the game, minimizing the quiz's length while maximizing its informative value, and embedding the pretest in the game. Our results are relevant for educational researchers and cybersecurity instructors of students at all learning levels.