Many Security Operations Centers (SOCs) today still heavily rely on signature-based Network Intrusion Detection Systems (NIDS) such as Suricata. The specificity of intrusion detection rules and the coverage provided by rulesets are common concerns within the professional community surrounding SOCs, which impact the effectiveness of automated alert post-processing approaches. We postulate a better understanding of factors influencing the quality of rules can help address current SOC issues. In this paper, we characterize the rules in use at a collaborating commercial (managed) SOC serving customers in sectors including education and IT management. During this process, we discover six relevant design principles, which we consolidate through interviews with experienced rule designers at the SOC.We then validate our design principles by quantitatively assessing their effect on rule specificity. We find that several of these design considerations significantly impact unnecessary workload caused by rules. For instance, rules that leverage proxies for detection, and rules that do not employ alert throttling or do not distinguish (un)successful malicious actions, cause significantly more workload for SOC analysts. Moreover, rules that match a generalized characteristic to detect malicious behavior, which is believed to increase coverage, also significantly increase workload, suggesting a tradeoff must be struck between rule specificity and coverage. We show that these design principles can be applied successfully at a SOC to reduce workload whilst maintaining coverage despite the prevalence of violations of the principles.

The investigations run by tier 1 (T1) analysts in a Security Operation Center are critical to the SOC operations as they represent the first gateway to alert escalation and incident response. Critically, they demand an accurate and as-complete-as-possible understanding of the events surrounding the investigated alert. This is a complex task inexperienced T1 analysts can easily lose track of. In this work, we collaborate with a commercial SOC to develop an alert investigation support tool to help inexperienced analysts identify and collect all the information relevant to the investigation of an alert. We evaluate the prototype tool with two qualitative studies. The first study employs T1 analysts from the SOC to evaluate the conformity of the tool to the underpinning analysis process. The second study employs 57 students, recruited from the same pool where the SOC acquires its junior analysts from, to evaluate whether it helps inexperienced analysts develop a complete understanding of events surrounding security alert data. Our findings suggest that employing the tool helps inexperienced analysts form a more accurate understanding of attacks, at no time cost. We discuss the wider implications for research and practice.