Intrusion detection systems (IDS) monitor system logs and network traffic to recognize malicious activities in computer networks. Evaluating and comparing IDSs with respect to their detection accuracies is thereby essential for their selection in specific use-cases. Despite a great need, hardly any labeled intrusion detection datasets are publicly available. As a consequence, evaluations are often carried out on datasets from real infrastructures, where analysts cannot control system parameters or generate a reliable ground truth, or private datasets that prevent reproducibility of results. As a solution, we present a collection of maintainable log datasets collected in a testbed representing a small enterprise. Thereby, we employ extensive state machines to simulate normal user behavior and inject a multi-step attack. For scalable testbed deployment, we use concepts from model-driven engineering that enable automatic generation and labeling of an arbitrary number of datasets that comprise repetitions of attack executions with variations of parameters. In total, we provide 8 datasets containing 20 distinct types of log files, of which we label 8 files for 10 unique attack steps. We publish the labeled log datasets and code for testbed setup and simulation online as open-source to enable others to reproduce and extend our results.

Evaluations of intrusion detection systems (IDS) require log datasets collected in realistic system environments. Existing testbeds therefore offer user simulations and attack scenarios that target specific use-cases. However, not only does the preparation of such testbeds require domain knowledge and time-consuming work, but also maintenance and modifications for other use-cases involve high manual efforts and repeated execution of tasks. In this article, we therefore propose to generate testbeds for IDS evaluation using strategies from model-driven engineering. In particular, our approach models system infrastructure, simulated normal behavior, and attack scenarios as testbed-independent modules. A transformation engine then automatically generates arbitrary numbers of testbeds, each with a particular set of characteristics and capable of running in parallel. Our approach greatly improves configurability and flexibility of testbeds and allows to reuse components across multiple scenarios. We use our proof-of-concept implementation to generate a labeled dataset for IDS evaluation that is published with this article.

Evaluations of intrusion detection systems (IDS) require log data sets collected in realistic system environments. Ex-isting testbeds therefore offer user simulations and attack scenarios that target specific use-cases. However, not only does the preparation of such testbeds require domain knowledge and time-consuming work, but also maintenance and modifications for other use-cases involve high manual efforts and repeated execution of tasks. We therefore propose to generate testbeds for IDS evaluation using strategies from model-driven engineering. In particular, our approach models system infrastructure, simulated normal behavior, and attack scenarios as testbed-independent modules. A transformation engine then automatically generates arbitrary numbers of testbeds, each with a particular set of characteristics and capable of running in parallel. Our approach greatly improves configurability and flexibility of testbeds and allows to reuse components across multiple scenarios. We use our proof-of-concept implementation to generate a labeled data set for IDS evaluation that is published with this paper.

Intrusion Detection Systems (IDS) rely on the availability and correctness of Indicators of Compromise (IoC), i.e., artifacts such as IP addresses that are known to correspond to malicious system activities. However, the simple nature and limited validity of these indicators impairs protection against cyber threats. Tactics, Techniques and Procedures (TTP) provide abstract information on attacker behavior, but are only available in human-readable format that prevents automatic detection using IDSs. In this paper we therefore propose an approach that extracts cyber threat intelligence from raw log data and combines the advantages of IoCs and TTPs by producing detectable patterns of complex system behavior. Other than existing approaches, our approach employs log data anomaly detection to disclose suspicious log events, which are used for iterative clustering, pattern recognition, and refinement. Our evaluations show that automatically extracted threat intelligence corresponding to a multi-step attack is suitable for detection of the same attack on another system.