Cybersecurity practitioners working in organisations implement risk controls aiming to improve the security of their systems. Determining prioritisation of the deployment of controls and understanding their likely impact on overall cybersecurity posture is challenging, yet without this understanding there is a risk of implementing inefficient or even harmful security practices. There is a critical need to comprehend the value of controls in reducing cyber-risk exposure in various organisational contexts, and the factors affecting their usage. Such information is important for research into cybersecurity risk and defences, for supporting cybersecurity decisions within organisations, and for external parties guiding cybersecurity practice such as standards bodies and cyber-insurance companies.

As cybersecurity breaches continue to increase in number and cost, and the demand for cyber-insurance rises, the ability to reason accurately about an organisation’s residual risk is of paramount importance. Security controls are integral to risk practice and decision-making: organisations deploy controls in order to reduce their risk exposure, and cyber-insurance companies provide coverage to these organisations based on their cybersecurity posture. Therefore, in order to reason about an organisation’s residual risk, it is critical to possess an accurate understanding of the controls organisations have in place and of the influence that these controls have on the likelihood that organisations will be harmed by a cyber-incident. Supporting evidence, however, for the effectiveness of controls is often lacking. With the aim of enriching internal threat data, in this article we explore a practical exercise in the form of a capture-the-flag (CTF) study. We experimented with a set of security controls and invited four professional penetration testers to solve the challenges. The results indicate that CTFs are a viable path for enriching threat intelligence and examining security controls, enabling us to begin to theorise about the relative effectiveness of certain risk controls on the face of threats, and to provide some recommendations for strengthening the cybersecurity posture.

Sonification (the representation of data as sound) may offer a solution to some of the network-security monitoring challenges faced in security operations centres (SOCs). Prior work has shown that sonification can present network-security information to humans effectively, and indicated that security practitioners foresee potential for sonification to aid in scenarios related to their work. The use of sonification by security practitioners in tasks relevant to SOCs has not been examined, however. To address this gap, we assessed the use of sonification by security practitioners in network-security monitoring tasks in an experimental setting. We report on the results of a study in which we compared the performance of security practitioners using a Security Information and Event Management (SIEM) tool with their performance using a SIEM tool that incorporated sonification, in a primary and a non-primary monitoring task. In both tasks, a number of aspects of the monitoring performance of participants were significantly improved when sonification was used. Our results support the potential for sonification to aid in SOC tasks, and indicate a need to validate the utility of sonification systems by running them in operational SOCs.