Hands-on training sessions become a standard way to develop and increase knowledge in cybersecurity. As practical cybersecurity exercises are strongly process-oriented with knowledge-intensive processes, process mining techniques and models can help enhance learning analytics tools. The design of our open-source analytical dashboard is backed by guidelines for visualizing multivariate networks complemented with temporal views and clustering. The design aligns with the requirements for post-training analysis of a special subset of cybersecurity exercises — supervised Capture the Flag games. Usability is demonstrated in a case study using trainees’ engagement measurement to reveal potential flaws in training design or organization.

Insider attacks in organizations are currently one of the most crucial cybersecurity challenges. Traitors are one of the most dangerous types of insider attackers. They are difficult to detect because they know the organization, processes, defense mechanisms, and employees. Besides abusing their rights and accesses, they can use their co-workers’ rights and accesses. They can hide their activities to perform attacks inconspicuously. The current masquerader detection techniques usually rely on the fact that the masquerader is an outsider, making them unable to detect attacks from the traitor. To detect these insider attackers, which we call masquerading traitors, we propose the human-as-solution approach and engage the employees in deciding whether the usage of their rights and accesses is suspicious and, therefore, might be performed by a masquerading traitor.

The hands-on cybersecurity training quality is crucial to mitigate cyber threats and attacks effectively. However, practical cybersecurity training is strongly process-oriented, making the post-training analysis very difficult. This paper presents process-mining methods applied to the learning analytics workflow. We introduce a unified approach to reconstructing behavioral graphs from sparse event logs of cyber ranges. Furthermore, we discuss significant data features that affect their practical usability for educational process mining. Based on that, methods of dealing with the complexity of process graphs are presented, taking advantage of the puzzle-based gamification of in-class training sessions.

Supervised Capture the Flag games represent a popular method of practical hands-on training in cybersecurity education. However, as cybersecurity training sessions are process-oriented, tutors have only a limited insight into what trainees are doing and how they deal with the tasks. From their perspective, it is necessary to have situational awareness, enabling them to identify and react to any issues during a training session as soon as they emerge. We propose a tool designed in collaboration with cybersecurity educators. Based on user requirements, we developed the Progress Visualization Tool, which provides educators with timely feedback through the session. More specifically, the tool informs educators of the training progression, helps identify the students who might struggle with their tasks, and reveals overall deviation from the schedule. We validated the tool through formative and summative qualitative in-lab evaluations. The participants appraised the impact on the training workflow and gave further insights regarding the tool. We discuss the insights and recommendations that arose from the evaluations as they could aid the design of future tools for supporting educators, not only of CTFs but also in other domains.

Hands-on training is an effective way to practice theoretical cybersecurity concepts and increase participants' skills. In this article, we discuss the application of visual analytics principles to the design, execution, and evaluation of training sessions. We propose a conceptual model employing visual analytics that supports the sensemaking activities of users involved in various phases of the training life cycle. The model emerged from our long-term experience in designing and organizing diverse hands-on cybersecurity training sessions. It provides a classification of visualizations and can be used as a framework for developing novel visualization tools supporting phases of the training life-cycle. We demonstrate the model application on examples covering two types of cybersecurity training programs.

Investigating cybersecurity incidents requires in-depth knowledge from the analyst. Moreover, the whole process is demanding due to the vast data volumes that need to be analyzed. While various techniques exist nowadays to help with particular tasks of the analysis, the process as a whole still requires a lot of manual activities and expert skills. We propose an approach that allows the analysis of disk snapshots more efficiently and with lower demands on expert knowledge. Following a user-centered design methodology, we implemented an analytical tool to guide analysts during security incident investigations. The viability of the solution was validated by an evaluation conducted with members of different security teams.

Capture the Flag games represent a popular method of cybersecurity training. Providing meaningful insight into the training progress is essential for increasing learning impact and supporting participants' motivation, especially in advanced hands-on courses. In this paper, we investigate how to provide valuable post-game feedback to players of serious cybersecurity games through interactive visualizations. In collaboration with domain experts, we formulated user requirements that cover three cognitive perspectives: gameplay overview, person-centric view, and comparative feedback. Based on these requirements, we designed two interactive visualizations that provide complementary views on game results. They combine a known clustering and time-based visual approaches to show game results in a way that is easy to decode for players. The purposefulness of our visual feedback was evaluated in a usability field study with attendees of the Summer School in Cyber Security. The evaluation confirmed the adequacy of the two visualizations for instant post-game feedback. Despite our initial expectations, there was no strong preference for neither of the visualizations in solving different tasks.

This Innovative Practice Full Paper addresses modern cyber ranges which represent unified platforms that offer efficient organization of complex hands-on exercises where participants can train their cybersecurity skills. However, the functionality targets mostly learners who are the primary users. Support of organizers performing analytic and evaluation tasks is weak and ad-hoc. It makes harder to improve the quality of an exercise, particularly its impact on learners. In this paper, we present an application of a well-structured visual analytics process to the organization of cyber exercises. We illustrate that the classification derived from the adoption of the visual analytics process helps to clarify and formalize analytical tasks of educators and enables their systematic support in cyber ranges. We demonstrate an application of our approach on a particular series of eight exercises we have organized in last three years. We believe the presented approach is beneficial for anyone involved in preparation and execution of any complex exercise.

Cyber defence exercises are intensive, hands-on learning events for teams of professionals who gain or develop their skills to successfully prevent and respond to cyber attacks. The exercises mimic the real-life, routine operation of an organization which is being attacked by an unknown offender. Teams of learners receive very limited immediate feedback from the instructors during the exercise; they can usually see only a scoreboard showing the aggregated gain or loss of points for particular tasks. An in-depth analysis of learners' actions requires considerable human effort, which results in days or weeks of delay. The intensive experience is thus not followed by proper feedback facilitating actual learning, and this diminishes the effect of the exercise.

We need more skilled cybersecurity professionals because the number of cyber threats and ingenuity of attackers is ever growing. Knowledge and skills required for cyber defence can be developed and exercised by lectures and lab sessions, or by active learning, which is seen as a promising and attractive alternative. In this paper, we present experience gained from the preparation and execution of cyber defence exercises involving various participants in a cyber range. The exercises follow a Red vs. Blue team format, in which the Red team conducts malicious activities against emulated networks and systems that have to be defended by Blue teams of learners. Although this exercise format is popular and used worldwide by numerous organizers in practice, it has been sparsely researched. We contribute to the topic by describing the general exercise life cycle, covering the exercise's development, dry run, execution, evaluation, and repetition. Each phase brings several challenges that exercise organizers have to deal with. We present lessons learned that can help organizers to prepare, run and repeat successful events systematically, with lower effort and costs, and avoid a trial-and-error approach that is often used.

The physical and cyber worlds are increasingly intertwined and exposed to cyber attacks. The KYPO cyber range provides complex cyber systems and networks in a virtualized, fully controlled and monitored environment. Time-efficient and cost-effective deployment is feasible using cloud resources instead of a dedicated hardware infrastructure. This paper describes the design decisions made during it’s development. We prepared a set of use cases to evaluate the proposed design decisions and to demonstrate the key features of the KYPO cyber range. It was especially cyber training sessions and exercises with hundreds of participants which provided invaluable feedback for KYPO platform development.