Cyber planners face a considerable challenge in finding holistic solutions for a cyber defence decision-support system - a core module of a cyber situation awareness capability. Due to a fast-evolving cyberspace, decision-makers assisted by technical staff are prone to carry out qualitative assessments when planning and conducting cyber operations instead of exclusively relying on quantitative assessments to articulate cyber defence mechanisms. A hybrid setting combining both types of assessments would be key to having the ability to monitor progression, anticipate deviations from initial plans, and evaluate effectiveness towards mission accomplishment. In line with this rationale, the authors propose a thorough analysis and tailoring of the military operation assessment framework applied to the characteristics of the cyberspace in view of identifying a proper methodology able to regularly assess the situation and provide mitigation measures to fix goal alignment problems, including measuring effects of cyber deception. Feedback mechanisms can capture the outcomes of implemented decisions, analyse discrepancies from expected results, and inform iterative improvements in the decision-support system. Such goals are considered operational objectives and decisive conditions of the operation design. The results are expected to shed some light about measuring the required performance of action and effectiveness using mission impact analysis and risk calculations among others. It remains to be further investigated how machine learning models could be integrated into the proposed hybrid system.

Cyber deception has emerged as a valuable technique in the field of cybersecurity, closely linked with adversarial Artificial Intelligence. In an era of pervasive automation, it is getting prominence as a research topic aimed at understanding how novel machine learning algorithms can be deceived using adversarial attacks that exploit vulnerabilities of their models. To this end, the paper describes the state-of-the-art of cyber deception for adversarial AI purposes, focusing on its benefits, challenges, and advanced techniques. In addition, this exploratory research attempts to extend its applicability to the fact that an appropriate and timely discovery of adversarial plans and associated actions may enhance own cyber resilience by introducing analytical findings of the adversary's intent into decision-making for cyber situational awareness. The study of adversarial thinking is as old as history and is one of the most relevant subjects rapidly incorporated into the operational planning process – a methodology to understand the operational environment. Adversarial knowledge is used for adapting own cyber defences in response to the cyber threat landscape.

The lack of suitable datasets and evaluation processes entails one of the most challenging gaps on the digital transformation era, where data-driven solutions like machine learning algorithms constitute a key pillar of the digitalization, virtualization and analytical on the emerging cyber-physical and ergonomic capabilities. This problem is even greater in the cyber defence domain, where for security or technical reasons, there is not data publicly or on-demand available concerning the role of the cyberspace on military operations. In this context, the expression popularized by the machine learning community "you go to the war with the data you have, not the data you might want" can be literally applied. In order to contribute to overcome this gap, this paper introduces CYSAS-S3, a novel dataset designed and created as the result of a research action that explores the principal needs on datasets by cyber commands, resulting in the generation of a collection of samples that correlated the impact of Advanced Persistent Threat (APT) behaviours and each phase of their cyber kill chain, regarding mission-level operations and goals.

The emerging need for cyber defence capabilities able to bring closer cyberspace supremacy in joint military operations has led defence practitioners to begin a cyber race where academy, researchers, industry and military organizations work together. Because of the higher maturity of civilian technologies for cybersecurity, this often involves adapting capabilities not initially intended for military use to new dual-use requirements, where concepts like operations, missions or Courses of Action (CoAs) shall be presented and aligned with the military doctrine. One of the main 'battle horses' in this transformation is to develop supporting systems able to facilitate the mission-centric acquisition of Cyber Situational Awareness (CSA), where the observations in the cyberspace shall be properly correlated, propagated and understood in the scope of planned/ongoing mission. But despite these CSA needs, there is a wide methodological gap in the lack of suitable validation and verification frameworks, which not only relies on the raising need for capabilities able to verify if the existing solutions meet the requirements to operate on military actions, but also to support the thorough development life-cycle of brand new cyber defence technologies. In these grounds, this research introduces a novel evaluation framework able to guide the evaluation of CSA related tools, for which three core validation concepts are discussed: Software, operational and application tests. They cover from the day-to-day implementation of the new capabilities, to their ability of facilitating that human decision-makers acquire a joint operational picture.