Improving cybersecurity education has become a priority for many countries and organizations worldwide. Computing societies and professional associations have recognized cybersecurity as a distinctive computing discipline and created specialized cybersecurity curricular guidelines. Higher education institutions are introducing new cybersecurity programs, attracting students to this expanding field. In this paper, we examined 101 study programs across 24 countries. Based on their analysis, we argue that top-ranked universities have not yet fully implemented the guidelines and offer programs that have "cyber" in their name but lack some essential elements of a cybersecurity program. In particular, most programs do not sufficiently cover non-technical components, such as law, policies, or risk management. Also, most programs teach knowledge and skills but do not expose students to experiential learning outside the traditional classroom (such as internships) to develop their competencies. As a result, graduates of these programs may not meet employer expectations and may require additional training. To help program directors and educators improve their programs and courses, this paper offers examples of effective practices from cybersecurity programs around the world and our teaching practice.

Diverse, accurate, and up-to-date training environments are essential for training cybersecurity experts and autonomous systems. However, preparation of their content is time-consuming and requires experts to provide detailed specifications. In this paper, we explore the challenges of automated generation of the content (composed of scenarios and terrains) for these environments.

Cyber threat modeling is a proactive method for identifying possible cyber attacks on network infrastructure that has a wide range of applications in security assessment, risk analysis, and threat exposure management. Popular modeling methods are kill chains and attack graphs. Kill chains divide attacks into phases, and attack graphs depict attack paths. A difficult issue is how to hierarchically model categories of cyber assets that should be used in threat models due to the variety of cyber systems in the current networks. This task should be addressed to provide automation of realistic threat modeling and interoperability with public knowledge bases, such as MITRE ATT&CK. In this paper, we propose a hierarchical modeling methodology for representing cyber assets in kill chain attack graphs. We illustrate its practical application on MITRE D3FEND’s Digital Artifact Ontology. Moreover, we define how cyber assets with related attack techniques should be transformed into logical facts and attack rules. We implemented proof-of-concept software modules that can process data obtained from network and host-based monitoring together with attack rules to generate attack graphs. We evaluated the approach with data from a cyber exercise captured in a network of a digital twin organization. The results show that the approach is applicable in real-world networks and can reveal ground-truth attacks.

This full paper in the research track evaluates the usage of data logged from cybersecurity exercises in order to predict students who are potentially at risk of performing poorly. Hands-on exercises are essential for learning since they enable students to practice their skills. In cybersecurity, hands-on exercises are often complex and require knowledge of many topics. Therefore, students may miss solutions due to gaps in their knowledge and become frustrated, which impedes their learning. Targeted aid by the instructor helps, but since the instructor's time is limited, efficient ways to detect struggling students are needed. This paper develops automated tools to predict when a student is having difficulty. We formed a dataset with the actions of 313 students from two countries and two learning environments: KYPO CRP and EDURange. These data are used in machine learning algorithms to predict the success of students in exercises deployed in these environments. After extracting features from the data, we trained and cross-validated eight classifiers for predicting the exercise outcome and evaluated their predictive power. The contribution of this paper is comparing two approaches to feature engineering, modeling, and classification performance on data from two learning environments. Using the features from either learning environment, we were able to detect and distinguish between successful and struggling students. A decision tree classifier achieved the highest balanced accuracy and sensitivity with data from both learning environments. The results show that activity data from cybersecurity exercises are suitable for predicting student success. In a potential application, such models can aid instructors in detecting struggling students and providing targeted help. We publish data and code for building these models so that others can adopt or adapt them.

For undergraduate students of computing, learning to solve complex practical problems in a team is an essential skill for their future careers. This skill is needed in various fields, such as in cybersecurity and IT governance. Tabletop exercises are an innovative teaching method used in practice for training teams in incident response and evaluation of contingency plans. However, tabletop exercises are not yet widely established in university education. This paper presents data and teaching experience from a cybersecurity course that introduces tabletop exercises in classrooms using a novel technology: INJECT Exercise Platform (IXP), a web-based learning environment for delivering and evaluating the exercises. This technology substantially improves the prior practice, since tabletop exercises worldwide have usually been conducted using pen and paper. Unlike in traditional tabletop exercises, which are difficult to evaluate manually, IXP provides insights into students' behavior and learning based on automated analysis of interaction data. We demonstrate IXP's capabilities and evolution by comparing exercise sessions hosted throughout three years at different stages of the platform's readiness. The analysis of student data is supplemented by the discussion of the lessons learned from employing IXP in computing education contexts. The data analytics enabled a detailed comparison of the teams' performance and behavior. Instructors who consider innovating their classes with tabletop exercises may use IXP and benefit from the insights in this paper.

Tabletop exercises are used to train personnel in the efficient mitigation and resolution of incidents. They are applied in practice to support the preparedness of organizations and to highlight inefficient processes. Since tabletop exercises train competencies required in the workplace, they have been introduced into computing courses at universities as an innovation, especially within cybersecurity curricula. To help computing educators adopt this innovative method, we survey academic publications that deal with tabletop exercises. From 140 papers we identified and examined, we selected 14 papers for a detailed review. The results show that the existing research deals predominantly with exercises that follow a linear format and exercises that do not systematically collect data about trainees' learning. Computing education researchers can investigate novel approaches to instruction and assessment in the context of tabletop exercises to maximize the impact of this teaching method. Due to the relatively low number of published papers, the potential for future research is immense. Our review provides researchers, tool developers, and educators with an orientation in the area, a synthesis of trends, and implications for further work.

Devices in computer networks cannot work without essential network services provided by a limited count of devices. Identification of device dependencies determines whether a pair of IP addresses is a dependency, i.e., the host with the first IP address is dependent on the second one. These dependencies cannot be identified manually in large and dynamically changing networks. Nevertheless, they are important due to possible unexpected failures, performance issues, and cascading effects. We address the identification of dependencies using a new approach based on graph-based machine learning. The approach belongs to link prediction based on a latent representation of the computer network’s communication graph. It samples random walks over IP addresses that fulfill time conditions imposed on network dependencies. The constrained random walks are used by a neural network to construct IP address embedding, which is a space that contains IP addresses that often appear close together in the same communication chain (i.e., random walk). Dependency embedding is constructed by combining values for IP addresses from their embedding and used for training the resulting dependency classifier. We evaluated the approach using IP flow datasets from a controlled environment and university campus network that contain evidence about dependencies. Evaluation concerning the correctness and relationship to other approaches shows that the approach achieves acceptable performance. It can simultaneously consider all types of dependencies and is applicable for batch processing in operational conditions.

As cyber threats endanger everyone, from regular users to computing professionals, spreading cybersecurity awareness becomes increasingly critical. Therefore, our university designed an innovative cybersecurity awareness course that is freely available online for students, employees, and the general public. The course offers simple, actionable steps that anyone can use to implement defensive countermeasures. Compared to other resources, the course not only suggests learners what to do, but explains why and how to do it. To measure the course impact, we administered it to 138 computer science undergraduates within a compulsory information security and cryptography course. They completed the course as a part of their homework and filled out a questionnaire after each lesson. Analysis of the questionnaire responses revealed that the students valued the course highly. They reported new learning, perspective changes, and transfer to practice. Moreover, they suggested suitable improvements to the course. Based on the results, we have distilled specific insights to help security educators design similar courses. Lessons learned from this study are relevant for cybersecurity instructors, course designers, and educational managers.

Fingerprinting a host's operating system is a very common yet precarious task in network, asset, and vulnerability management. Estimating the operating system via network traffic analysis may leverage TCP/IP header parameters or complex analysis of hosts' behavior using machine learning. However, the existing approaches are becoming obsolete as network traffic evolves which makes the problem still open. This paper discusses various approaches to passive OS fingerprinting and their evolution in the past twenty years. We illustrate their usage, compare their results in an experiment, and list challenges faced by the current fingerprinting approaches. The hosts' differences in network stack settings were initially the most important information source for OS fingerprinting, which is now complemented by hosts' behavioral analysis and combined approaches backed by machine learning. The most impactful reasons for this evolution were the Internet-wide network traffic encryption and the general adoption of privacy-preserving concepts in application protocols. Other changes, such as the increasing proliferation of web applications on handheld devices, raised the need to identify these devices in the networks, for which we may use the techniques of OS fingerprinting.

Cybersecurity training is a key endeavour for ensuring that the IT workforce possess the knowledge and practical skills required to counter the ever-increasing cybersecurity threats that our society is faced with. While some related systems, such as Capture The Flag platforms, have been available for almost one decade, platforms that support full-fledged cybersecurity training exercises have only been released as open source in recent years. Given the complexity of such cybersecurity training platforms, the question that arises is how to meaningfully evaluate and compare their capabilities in order to identify the most suitable solution for a given type of organization and/or training activity. In this paper, we introduce a capability assessment methodology for cybersecurity training platforms that focuses on the three key aspects of training: content representation, environment management, and training facilitation. The assessment tool that we developed is used to evaluate two open-source cybersecurity training platforms, CyTrONE and KYPO. We then conduct a comparative analysis of these two platforms based on our first-hand developer experience with them, and discuss the lessons learned from implementing, deploying and using these platforms. The assessment tool and the detailed technical comparative analysis that we conducted are intended as instruments and references for anyone who plans to deploy or develop cybersecurity training platforms.

Identification of cyber threats is one of the essential tasks for security teams. Currently, cyber threats can be identified using knowledge organized into various formats, enumerations, and knowledge bases. This paper studies the current challenges of identifying vulnerabilities and threats in cyberspace using enumerations and data about assets. Although enumerations are used in practice, we point out several issues that still decrease the quality of vulnerability and threat identification. Since vulnerability identification methods are based on network monitoring and agents, the issues are related to the asset discovery, the precision of vulnerability discovery, and the amount of data. On the other hand, threat identification utilizes graph-based, nature-language, machine-learning, and ontological approaches. The current trend is to propose methods that utilize tactics, techniques, and procedures instead of low-level indicators of compromise to make cyber threat identification more mature. Cooperation between standards from threat, vulnerability, and asset management is also an unresolved issue confirmed by analyzing relationships between public enumerations and knowledge bases. Last, we studied the usability of techniques from the MITRE ATT&CK knowledge base for threat modeling using network monitoring to capture data. Although network traffic is not the most used data source, it allows the modeling of almost all tactics from the MITRE ATT&CK.

This Research To Practice Full Paper presents an instructor guide and a tool to improve the creation of cybersecurity hands-on training with adaptive learning support. Adaptive learning uses students' performance and skills to assign suitable tasks to improve their learning experience. While it is well-established in many domains, it is rarely used in operating systems, networking, and cybersecurity. In this paper, we improve and present how to ease the creation and optimization process of adaptive hands-on training by instructors. To the best of our knowledge, this paper is one of the first works investigating the process of creating cybersecurity training with adaptive learning. The training uses metrics such as pre-training assessment and performance during the previous tasks in training to assign suitable tasks for each student. With the help of the developed tool, we demonstrate how metrics settings influence the students' transitions between training tasks. The instructors can easily visualize students' transitions throughout the training. This approach helps the instructors adapt the metrics to predict students' transitions between tasks for each training session. The results from performed simulations show that our tool might increase the efficiency of the adaptive training and students' experience even more. Using the experience from the simulations and past training sessions, we propose the design process for the whole creation of adaptive training. This design process is general enough to be adopted by other domains such as operating systems and networking that may use adaptive learning techniques for their hands-on assignments. We have released the tool and all the software components under an open-source license, so other instructors can freely use and adopt them.

The ever-evolving capabilities of cyber attackers force security administrators to focus on the early identification of emerging threats. Targeted cyber attacks usually consist of several phases, from initial reconnaissance of the network environment to final impact on objectives. This paper investigates the identification of multi-step cyber threat scenarios using kill chain and attack graphs. Kill chain and attack graphs are threat modeling concepts that enable determining weak security defense points. We propose a novel kill chain attack graph that merges kill chain and attack graphs together. This approach determines possible chains of attacker’s actions and their materialization within the protected network. The graph generation uses a categorization of threats according to violated security properties. The graph allows determining the kill chain phase the administrator should focus on and applicable countermeasures to mitigate possible cyber threats. We implemented the proposed approach for a predefined range of cyber threats, especially vulnerability exploitation and network threats. The approach was validated on a real-world use case. Publicly available implementation contains a proof-of-concept kill chain attack graph generator.

Achieving situational awareness is a challenging process in current HTTPS-dominant web traffic. In this paper, we propose a new approach to encrypted web traffic monitoring. First, we design a method for correlating host-based and network monitoring data based on their common features and a correlation time-window. Then we analyze the correlation results in detail to identify configurations of web servers and monitoring infrastructure that negatively affect the correlation. We describe these properties and possible data preprocessing techniques to minimize their impact on correlation performance. Furthermore, to test the correlation method’s behavior in different web server setups and for recent encryption protocols, we modify it by adapting the correlation features to TLS 1.3 and QUIC. Finally, we evaluate the correlation method on a dataset collected from a campus network. The results show that while the correlation requires monitoring of custom event and flow features, it remains feasible even when using encryption protocols designed for the near future.

Cybersecurity students need to develop practical skills such as using command-line tools. Hands-on exercises are the most direct way to assess these skills, but assessing students' mastery is a challenging task for instructors. We aim to alleviate this issue by modeling and visualizing student progress automatically throughout the exercise. The progress is summarized by graph models based on the shell commands students typed to achieve discrete tasks within the exercise. We implemented two types of models and compared them using data from 46 students at two universities. To evaluate our models, we surveyed 22 experienced computing instructors and qualitatively analyzed their responses. The majority of instructors interpreted the graph models effectively and identified strengths, weaknesses, and assessment use cases for each model. Based on the evaluation, we provide recommendations to instructors and explain how our graph models innovate teaching and promote further research. The impact of this paper is threefold. First, it demonstrates how multiple institutions can collaborate to share approaches to modeling student progress in hands-on exercises. Second, our modeling techniques generalize to data from different environments to support student assessment, even outside the cybersecurity domain. Third, we share the acquired data and open-source software so that others can use the models in their classes or research.

Networking, operating systems, and cybersecurity skills are exercised best in an authentic environment. Students work with real systems and tools in a lab environment and complete assigned tasks. Since all students typically receive the same assignment, they can consult their approach and progress with an instructor, a tutoring system, or their peers. They may also search for information on the Internet. Having the same assignment for all students in class is standard practice efficient for learning and developing skills. However, it is prone to cheating when used in a summative assessment such as graded homework, a mid-term test, or a final exam. Students can easily share and submit correct answers without completing the assignment. In this paper, we discuss methods for automatic problem generation for hands-on tasks completed in a computer lab environment. Using this approach, each student receives personalized tasks. We developed software for generating and submitting these personalized tasks and conducted a case study. The software was used for creating and grading a homework assignment in an introductory security course enrolled by 207 students. The software revealed seven cases of suspicious submissions, which may constitute cheating. In addition, students and instructors welcomed the personalized assignments. Instructors commented that this approach scales well for large classes. Students rarely encountered issues while running their personalized lab environment. Finally, we have released the open-source software to enable other educators to use it in their courses and learning environments.

This Research To Practice Full Paper presents how learning experience influences students' capability to learn and their motivation for further learning. Although each student is different, standard instruction methods do not adapt to individual students. Adaptive learning reverses this practice and attempts to improve the student experience. While adaptive learning is well-established in programming, it is rarely used in cybersecurity education. This paper is one of the first works investigating adaptive learning in cybersecurity training. First, we analyze the performance of 95 students in 12 training sessions to understand the limitations of the current training practice. Less than half of the students (45 out of 95) completed the training without displaying any solution, and only in two sessions, all students completed all phases. Then, we simulate how students would proceed in one of the past training sessions if it would offer more paths of various difficulty. Based on this simulation, we propose a novel tutor model for adaptive training, which considers students' proficiency before and during an ongoing training session. The proficiency is assessed using a pre-training questionnaire and various in-training metrics. Finally, we conduct a case study with 24 students and new training using the proposed tutor model and adaptive training format. The results show that the adaptive training does not overwhelm students as the original static training format. In particular, adaptive training enables students to enter several alternative training phases with lower difficulty than the phases in the original training. The proposed adaptive format is not restricted to particular training used in our case study. Therefore, it can be applied to practicing any cybersecurity topic or even in other related computing fields, such as networking or operating systems. Our study indicates that adaptive learning is a promising approach for improving the student experience in cybersecurity education. We also highlight diverse implications for educational practice that improve students' experience.

This Full Paper in the Innovative Practice category presents and evaluates a technical innovation for hands-on classes. When learning cybersecurity, operating systems, or networking, students perform practical tasks using a broad range of command-line tools. Collecting and analyzing data about the command usage can reveal valuable insights into how students progress and where they make mistakes. However, few learning environments support recording and inspecting command-line inputs, and setting up an efficient infrastructure for this purpose is challenging. To aid engineering and computing educators, we share the design and implementation of an open-source toolset for logging commands that students execute on Linux machines. Compared to basic solutions, such as shell history files, the toolset's novelty and added value are threefold. First, its configuration is automated so that it can be easily used in classes on different topics. Second, it collects metadata about the command execution, such as a timestamp, hostname, and IP address. Third, all data are instantly forwarded to central storage in a unified, semi-structured format. This enables automated processing of the data, both in real-time and post hoc, to enhance the instructors' understanding of student actions. The toolset works independently of the teaching content, the training network's topology, or the number of students working in parallel. We demonstrated the toolset's value in two learning environments at four training sessions. Over two semesters, 50 students played educational cybersecurity games using a Linux command-line interface. Each training session lasted approximately two hours, during which we recorded 4439 shell commands. The semiautomated data analysis revealed different solution patterns, used tools, and misconceptions of students. Our insights from creating the toolset and applying it in teaching practice are relevant for instructors, researchers, and developers of learning environments. We provide the software and data resulting from this work so that others can use them in their hands-on classes.

This Innovative Practice full paper describes a technical innovation for scalable teaching of cybersecurity hands-on classes using interactive learning environments. Hands-on experience significantly improves the practical skills of learners. However, the preparation and delivery of hands-on classes usually do not scale. Teaching even small groups of students requires a substantial effort to prepare the class environment and practical assignments. Further issues are associated with teaching large classes, providing feedback, and analyzing learning gains. We present our research effort and practical experience in designing and using learning environments that scale up hands-on cybersecurity classes. The environments support virtual networks with full-fledged operating systems and devices that emulate realworld systems. The classes are organized as simultaneous training sessions with cybersecurity assignments and learners' assessment. For big classes, with the goal of developing learners' skills and providing formative assessment, we run the environment locally, either in a computer lab or at learners' own desktops or laptops. For classes that exercise the developed skills and feature summative assessment, we use an on-premises cloud environment. Our approach is unique in supporting both types of deployment. The environment is described as code using open and standard formats, defining individual hosts and their networking, configuration of the hosts, and tasks that the students have to solve. The environment can be repeatedly created for different classes on a massive scale or for each student on-demand. Moreover, the approach enables learning analytics and educational data mining of learners' interactions with the environment. These analyses inform the instructor about the student's progress during the class and enable the learner to reflect on a finished training. Thanks to this, we can improve the student class experience and motivation for further learning. Using the presented environments KYPO Cyber Range Platform and Cyber Sandbox Creator, we delivered the classes on-site or remotely for various target groups of learners (K-12, university students, and professional learners). The learners value the realistic nature of the environments that enable exercising theoretical concepts and tools. The instructors value time-efficiency when preparing and deploying the hands-on activities. Engineering and computing educators can freely use our software, which we have released under an open-source license. We also provide detailed documentation and exemplary hands-on training to help other educators adopt our teaching innovations and enable sharing of reusable components within the community.

This paper is a digest of the thesis on predicting cyber attacks in a collaborative environment. While previous works mostly focused on predicting attacks as seen from a single observation point, we proposed taking advantage of collaboration and exchange of intrusion detection alerts among organizations and networks. Thus, we can observe the cyber attack on a large scale and predict the next action of an adversary and its target. The thesis follows the three levels of cyber situational awareness: perception, comprehension, and projection. In the perception phase, we discuss the improvements of intrusion detection systems that allow for sharing intrusion detection alerts and their correlation. In the comprehension phase, we employed data mining to discover frequent attack patterns. In the projection phase, we present the analytical framework for the predictive analysis of the alerts backed by data mining and contemporary data processing approaches. The results are shown from experimental evaluation in the security alert sharing platform SABU, where real-world alerts from Czech academic and commercial networks are shared. The thesis is accompanied by the implementation of the analytical framework and a dataset that provides a baseline for future work.

Cyber situation awareness has been recognized as a vital requirement for effective cyber defense. Cyber situation awareness allows cybersecurity operators to identify, understand, and anticipate incoming threats. Achieving and maintaining the cyber situation awareness is a challenging task given the continuous evolution of the computer networks, increasing volume and speeds of the data in a network, and rising number of threats to network security. Our work contributes to the continuous evolution of cyber situation awareness by the research of novel approaches to the perception and comprehension of a computer network. We concentrate our research efforts on the domain of IP flow network monitoring. We propose improvements to the IP flow monitoring techniques that enable the enhanced perception of a computer network. Further, we conduct detailed analyses of network traffic, which allows for an in-depth understanding of host behavior in a computer network. Last but not least, we propose a novel approach to IP flow network monitoring that enables real-time cyber situation awareness.

Asset identification plays a vital role in situational awareness building. However, the current trends in communication encryption and the emerging new protocols turn the well-known methods into a decline as they lose the necessary data to work correctly. In this paper, we examine the traffic patterns of the TLS protocol and its changes introduced in version 1.3. We train a machine learning model on TLS handshake parameters to identify the operating system of the client device and compare its results to well-known identification methods. We test the proposed method in a large wireless network. Our results show that precise operating system identification can be achieved in encrypted traffic of mobile devices and notebooks connected to the wireless network.

There are different requirements on cybersecurity of industrial control systems and information technology systems. This fact exacerbates the global issue of hiring cybersecurity employees with relevant skills. In this paper, we present KYPO4INDUSTRY training facility and a course syllabus for beginner and intermediate computer science students to learn cybersecurity in a simulated industrial environment. The training facility is built using open-source hardware and software and provides reconfigurable modules of industrial control systems. The course uses a flipped classroom format with hands-on projects: the students create educational games that replicate real cyber attacks. Throughout the semester, they learn to understand the risks and gain capabilities to respond to cyber attacks that target industrial control systems. Our described experience from the design of the testbed and its usage can help any educator interested in teaching cybersecurity of cyber-physical systems.

Cybersecurity is now more important than ever, and so is education in this field. However, the cybersecurity domain encompasses an extensive set of concepts, which can be taught in different ways and contexts. To understand the state of the art of cybersecurity education and related research, we examine papers from the ACM SIGCSE and ACM ITiCSE conferences. From 2010 to 2019, a total of 1,748 papers were published at these conferences, and 71 of them focus on cybersecurity education. The papers discuss courses, tools, exercises, and teaching approaches. For each paper, we map the covered topics, teaching context, evaluation methods, impact, and the community of authors. We discovered that the technical topic areas are evenly covered (the most prominent being secure programming, network security, and offensive security), and human aspects, such as privacy and social engineering, are present as well. The interventions described in SIGCSE and ITiCSE papers predominantly focus on tertiary education in the USA. The subsequent evaluation mostly consists of collecting students' subjective perceptions via questionnaires. However, less than a third of the papers provide supplementary materials for other educators, and none of the authors published their dataset. Our results provide orientation in the area, a synthesis of trends, and implications for further research. Therefore, they are relevant for instructors, researchers, and anyone new in the field of cybersecurity education. The information we collected and synthesized from individual papers are organized in a publicly available dataset.

Capture the Flag games are software applications designed to exercise cybersecurity concepts, practice using security tools, and understand cyber attacks and defense. We develop and employ these games at our university for training purposes, unlike in the traditional competitive setting. During the gameplay, it is possible to collect data about players' in-game actions, such as typed commands or solution attempts, including the timing of these actions. Although such data was previously employed in computer security research, to the best of our knowledge, there were few attempts to use this data primarily to improve education. In particular, we see an open and challenging research problem in creating an artificial intelligence assistant that would facilitate the learning of each player. Our goal is to propose, apply, and experimentally evaluate data analysis and machine learning techniques to derive information about the players' interactions from the in-game data. We want to use this information to automatically provide each player with a personalized formative assessment. Such assessment will help the players identify their mastered concepts and areas for improvement, along with suggestions and actionable steps to take. Furthermore, we want to identify high- or low-performing players during the game, and subsequently, offer them game tasks more suitable to their skill level. These interventions would supplement or even replace feedback from instructors, which would significantly increase the learning impact of the games, enable more students to learn cybersecurity skills at an individual pace, and lower the costs.

In this short paper, we take a first step towards empirically assessing Internet-wide malicious activities generated from and targeted towards Internet-scale business sectors (i.e., financial, health, education, etc.) and critical infrastructure (i.e., utilities, manufacturing, government, etc.). Facilitated by an innovative and a collaborative large-scale effort, we have conducted discussions with numerous Internet entities to obtain rare and private information related to allocated IP blocks pertaining to the aforementioned sectors and critical infrastructure. To this end, we employ such information to attribute Internet-scale maliciousness to such sectors and realms, in an attempt to provide an in-depth analysis of the global cyber situational posture. We draw upon close to 16.8 TB of darknet data to infer probing activities (typically generated by malicious/infected hosts) and DDoS backscatter, from which we distill IP addresses of victims. By executing week-long measurements, we observed an alarming number of more than 11,000 probing machines and 300 DDoS attack victims hosted by critical sectors. We also generate rare insights related to the maliciousness of various business sectors, including financial, which typically do not report their hosted and targeted illicit activities for reputation-preservation purposes. While we treat the obtained results with strict confidence due to obvious sensitivity reasons, we postulate that such generated cyber threat intelligence could be shared with sector/critical infrastructure operators, backbone networks and Internet service providers to contribute to the overall threat remediation objective.

In today's complex computer networks, we are constantly facing a risk of data loss, system compromise, or intellectual property theft. The complexity of the networks hinders their effective defense. A Network-wide Cyber Situational Awareness (NwCSA) has been introduced to assist a network security administrator with network security. The concept, how-ever, faces several challenges that hinder an efficient application of the NwCSA in a real-world environment. The challenges include the overload of raw data, low speed of reaction, and a lack of context and unified view on a network. In this paper, we present a novel framework that faces above mentioned challenges. The framework leverages a distributed data stream processing system and methods for real-time big data processing. The framework is evaluated with respect to stated requirements on systems for NwCSA. Moreover, we present a prototype framework implementation and provide lessons learned from its real-world deployment.

Operating system fingerprinting methods are well- known in the domain of static networks and managed environments. Yet few studies tackled this challenge in real networks, where users can bring and connect any device. We evaluate the performance of three OS fingerprinting methods on a large dataset collected from university wireless network. Our results show that method based on HTTP User-agents is the most accurate but can identify only low portion of the traffic. TCP/IP parameters method proved to be the opposite with high coverage but low accuracy. We also implemented a new method based on detection of communication to OS-specific domains. Its performance is comparable to the two established ones. Next, we discuss the impacts of traffic encryption and embracing new protocols such as IPv6 or HTTP/2.0 on OS fingerprinting. Our findings suggest that OS identification based on specific domain detection is viable and corresponds to the current directions of network traffic evolution, while methods based on TCP/IP parameters and User-agents will become ineffective in the future.

Data mining is well-known for its ability to extract concealed and indistinct patterns in the data, which is a common task in the field of cyber security. However, data mining is not always used to its full potential among cyber security community. In this paper, we discuss usability of sequential pattern and rule mining, a subset of data mining methods, in an analysis of cyber security alerts. First, we survey the use case of data mining, namely alert correlation and attack prediction. Subsequently, we evaluate sequential pattern and rule mining methods to find the one that is both fast and provides valuable results while dealing with the peculiarities of security alerts. An experiment was performed using the dataset of real alerts from an alert sharing platform. Finally, we present lessons learned from the experiment and a comparison of the selected methods based on their performance and soundness of the results.

We need more skilled cybersecurity professionals because the number of cyber threats and ingenuity of attackers is ever growing. Knowledge and skills required for cyber defence can be developed and exercised by lectures and lab sessions, or by active learning, which is seen as a promising and attractive alternative. In this paper, we present experience gained from the preparation and execution of cyber defence exercises involving various participants in a cyber range. The exercises follow a Red vs. Blue team format, in which the Red team conducts malicious activities against emulated networks and systems that have to be defended by Blue teams of learners. Although this exercise format is popular and used worldwide by numerous organizers in practice, it has been sparsely researched. We contribute to the topic by describing the general exercise life cycle, covering the exercise's development, dry run, execution, evaluation, and repetition. Each phase brings several challenges that exercise organizers have to deal with. We present lessons learned that can help organizers to prepare, run and repeat successful events systematically, with lower effort and costs, and avoid a trial-and-error approach that is often used.

The physical and cyber worlds are increasingly intertwined and exposed to cyber attacks. The KYPO cyber range provides complex cyber systems and networks in a virtualized, fully controlled and monitored environment. Time-efficient and cost-effective deployment is feasible using cloud resources instead of a dedicated hardware infrastructure. This paper describes the design decisions made during it’s development. We prepared a set of use cases to evaluate the proposed design decisions and to demonstrate the key features of the KYPO cyber range. It was especially cyber training sessions and exercises with hundreds of participants which provided invaluable feedback for KYPO platform development.

In this paper, we describe a network defence strategy testbed, which could be utilized for testing the strategy decision logic against simulated attacks or real attackers. The testbed relies on a network of honeypots and the high level of logging and monitoring the honeypots provide. Its main advantage is that only the decision logic implementation is needed in order to test the strategy. The testbed also evaluates the tested network defence strategy. We demonstrate an example of network defence strategy implementation, the test setup, progress, and results. The source code of the testbed is available on GitHub.

Modern distributed stream processing systems can potentially be applied to real time network flow processing. However, differences in performance make some systems more suitable than others for being applied to this domain. We propose a novel performance benchmark, which is based on common security analysis algorithms of NetFlow data to determine the suitability of distributed stream processing systems. Three of the most used distributed stream processing systems are bench-marked and the results are compared with NetFlow data processing challenges and requirements. The benchmark results show that each system reached a sufficient data processing speed using a basic deployment scenario with little to no configuration tuning. Our benchmark, unlike any other, enables the performance of small structured messages to be processed on any stream processing system.

Performing research on live network traffic requires the traffic to be well documented and described. The results of such research are heavily dependent on the particular network. This paper presents a study of network characteristics, which can be used to describe the behaviour of a network. We propose a number of characteristics that can be collected from the networks and evaluate them on five different networks of Masaryk University. The proposed characteristics cover IP, transport and application layers of the network traffic. Moreover, they reflect strong day-night and weekday patterns that are present in most of the networks. Variation in the characteristics between the networks indicates that they can be used for the description and differentiation of the networks. Furthermore, a weak correlation between the chosen characteristics implies their independence and contribution to network description.

The encryption of network traffic complicates legitimate network monitoring, traffic analysis, and network forensics. In this paper, we present real-time lightweight identification of HTTPS clients based on network monitoring and SSL/TLS fingerprinting. Our experiment shows that it is possible to estimate the User-Agent of a client in HTTPS communication via the analysis of the SSL/TLS handshake. The fingerprints of SSL/TLS handshakes, including a list of supported cipher suites, differ among clients and correlate to User-Agent values from a HTTP header. We built up a dictionary of SSL/TLS cipher suite lists and HTTP User-Agents and assigned the User-Agents to the observed SSL/TLS connections to identify communicating clients. The dictionary was used to classify live HTTPS network traffic. We were able to retrieve client types from 95.4 % of HTTPS network traffic. Further, we discussed host-based and network-based methods of dictionary retrieval and estimated the quality of the data.

The growing share of encrypted network traffic complicates network traffic analysis and network forensics. In this paper, we present real-time lightweight identification of HTTPS clients based on network monitoring and SSL/TLS fingerprinting. Our experiment shows that it is possible to estimate the User-Agent of a client in HTTPS communication via the analysis of the SSL/TLS handshake. The fingerprints of SSL/TLS handshakes, including a list of supported cipher suites, differ among clients and correlate to User-Agent values from a HTTP header. We built up a dictionary of SSL/TLS cipher suite lists and HTTP User-Agents and assigned the User-Agents to the observed SSL/TLS connections to identify communicating clients. We discuss host-based and network-based methods of dictionary retrieval and estimate the quality of the data. The usability of the proposed method is demonstrated on two case studies of network forensics.

Digital home business is a large potential market for which ACEMIND European collaborative project intends to provide a set of consistent solutions to enhance the local network management in home and small enterprises. These networks are typically constituted by some tens of devices connected to each other via a set of wired and wireless technologies. The deployment of such networks has been intensive in Europe in the last decade and will continue in the future. However common end-users still suffer from complicated installation, operation or maintenance. Recent progress has been made to integrate heterogeneous connectivity into a single network. A proof of concept has been shown thanks to European collaborative project OMEGA and the hybrid network demonstrator has led to an IEEE standard and available commercial products. The goal of ACEMIND project is to enrich this initial concept with new features and services such as energy efficiency, packets load balancing or anomaly detection. Another target is to integrate future products using new transmission technologies. Then, using a unique user-friendly GUI (Graphical User Interface), it will be possible to monitor and manage the home network with a large range of services such as energy optimization, UHD TV or wellness.

In this paper we present a cloud-based research testbed designed to aid network security managers. The testbed enables operators to emulate various network topologies, services, and to analyze attacks threatening these systems. A possibility to test results of network management measures is desired, since testing these measures in a production environment is always not possible. We demonstrate a testbed use case, which aids to scrutinize network behavior under attack. Our use case is based on a large DDoS attack which targeted network infrastructure and web servers in Czech Republic in March, 2013.

Malware targeting broadband devices like ADSL modems, routers and wireless access points is very frequent in recent days. In this paper, we provide a formal description of modem malware life cycle. Furthermore, we propose a set of techniques to perform detailed analysis of infected modem and we provide the binary samples of modem malware at our web repository. Description of the modem malware evolution is also included. Based on our experiences with analysing and monitoring modem malware, we report on long-term statistics of modem malware activities in campus network including a discovery of new botnet. We propose NetFlow based detection method to reveal the modem malware spreading.