Computer science experimentation, whether it be for safety, reliability or cybersecurity, is an important part of scientific advancement. Evaluation of relative merits of various experiments typically requires well-calibrated benchmarks that can be used to measure the experimental results. This paper reviews current trends in using benchmarks in fuzzing experimental research for cybersecurity, specifically with metrics related to coverage analysis. Strengths and weaknesses of the current techniques are evaluated and suggestions for improving the current approaches are proposed. The end goal is to convince researchers that benchmarks for experimentation must be well documented, archived and calibrated so that the community knows how well the tools and techniques perform with respect to the possible maximum in the benchmark.

Automated cyber defense tools require the ability to analyze binary applications, detect vulnerabilities and automatically patch those vulnerabilities. The insertion of security mechanisms that operate at function boundaries (e.g, control flow mitigation, stack guards) require automated detection of those boundaries. This paper introduces a publicly available function boundary detection tool for 32 and 64-bit Intel binaries running under Linux, that is more accurate than other reported approaches.

Due to the characteristics and connectivity of today's Cyber-Physical Control Systems (CPCS) and critical infrastructures, cyber-attacks on these systems are currently difficult to prevent in an efficient and sustainable manner. Prevention and mitigation need accurate identification and evaluation of: system vulnerabilities, likely threats and attacks, and applicable hardening measures. Furthermore, the ability to prioritize hardening measures based on accurate assessments of threat risk and consequence and mitigation availability, applicability, and cost is also needed. To address this challenge we created HESTIA: High-level and Extensible System for Training and Infrastructure risk Assessment. In this paper, we describe the latest architecture and working principles of HESTIA. When fully developed, the HESTIA process and tool-set will enable CPCS engineers to, iteratively: 1) specify a CPCS, 2) select applicable attacks and hardening measures from a library, 3) check specifications for consistency and applicability, and 4) merge attack and hardening specifications into a new CPCS model. In addition, we add support for device specification templates. HESTIA enables the discovery of attack-defend scenarios through simulation and the design of optimal hardening strategies for a given CPCS. This paper is a shortened and updated version of a journal article entitled An architecture for HESTIA to appear in the International Journal of Internet of Things and Cyber-Assurance.

Many computational intelligence techniques for anomaly based network intrusion detection can be found in literature. Translating a newly discovered intrusion recognition criteria into a distributable rule can be a human intensive effort. This paper explores a multi-modal genetic algorithm solution for autonomous rule creation. This algorithm focuses on the process of creating rules once an intrusion has been identified, rather than the evolution of rules to provide a solution for intrusion detection. The algorithm was demonstrated on anomalous ICMP network packets (input) and Snort rules (output of the algorithm). Output rules were sorted according to a fitness value and any duplicates were removed. The experimental results on ten test cases demonstrated a 100 percent rule alert rate. Out of 33,804 test packets 3 produced false positives. Each test case produced a minimum of three rule variations that could be used as candidates for a production system.

We introduce TG/MC, a Monte Carlo approach for evaluating the impact of uncertainty about vulnerabilities upon forecasts of security for a real-world system modeled by a protection graph. A TG/MC model defines a vulnerability as a potential change to an otherwise safe initial protection graph that, if exploited, leads to an unauthorized state, a violation of the system's security policy through the application of TG rules. TG/MC captures uncertainties about vulnerabilities as probability distributions and forecasts the probability of a specific security violation. TG/MC extends beyond the rigid yes/no analysis of safety in a TG protection graph to consider uncertainty in questions of security for real-world systems.

System security involves decisions in at least three areas: identification of well-defined security policies, selection of cost-effective defence strategies, and implementation of real-time defence tactics. Although choices made in each of these areas affect the others, existing decision models typically handle these three decision areas in isolation. There is no comprehensive tool that can integrate them to provide a single efficient model for safeguarding a network. In addition, there is no clear way to determine which particular combinations of defence decisions result in cost-effective solutions. To address these problems, this paper introduces a Layered Decision Model (LDM) for use in deciding how to address defence decisions based on their cost-effectiveness. To validate the LDM and illustrate how it is used, we used simulation to test model rationality and applied the LDM to the design of system security for an e-commercial business case.

The successful design and implementation of secure systems must include security concerns from the beginning. A component that processes data at multiple security levels is critical and must go through additional evaluation to ensure the processing i

The most efficient contributory group key agreement protocols conduct their operations using a tree-based structure to guide communication and computation. Existing performance comparisons of group key protocols have only evaluated the cost of single operations in isolation. This paper expands this work by evaluating the performance impact of tree rebalancing operations on the overall cost of a group key agreement protocol over the course of the execution of several operations

The successful design and implementation of secure systems must occur from the beginning. A component that must process data at multiple security levels is very critical and must go through additional evaluation to ensure the processing is secure. It is common practice to isolate and separate the processing of data at different levels into different components. In this paper we present architecture-based refinement techniques for the design of multilevel secure systems. We discuss what security requirements must be satisfied through the refinement process, including when separation works and when it does not. The process oriented approach will lead to verified engineering techniques for secure systems, which should greatly reduce the cost of certification of those systems.

Critical and catastrophic failures in high assurance and critical computing systems can arise from unfounded assumptions of independence between system components, requirements, and constraints (work product sections), which can stem from misunderstandings and miscommunication between system engineers, managers, and operators and from inadequate or incomplete traceability between system work products. In this article, we propose a formal framework for the effective implementation of traceability between work product sections along with a technique for discovering potential causes of critical failures in high assurance and critical computing system models. We introduce a new abstraction of interrelated work product sections called implementation meta-work product and describe how our technique finds these meta-work products. We also demonstrate how this technique can be used to help analysts discover potential causes of safety-related errors in high assurance and critical computing systems by applying it to one case study of a known critical error and to one case study where we anticipate potential safety hazards.

This panel addressed the use of computer diversity as a strategy for computer security. It is our view that there are significant knowledge gaps in the science underlying diversity as a computer defense mechanism which hinders its usefulness. These gaps include the true cost of diversity, a lack of metrics for diversity and the trade offs between diversity and other defensive strategies. We also wanted to highlight on-going diversity research from other disciplines which could potentially be applied to diversity for computer security.Four panelists were selected based on their experience with diversity within the context of computer security or other research areas. The panelists' backgrounds include biology, software reliability, security and dependable systems. Each panelist presented a statement which was discussed by NSPW participants.The discussion was lively and informative both during and after the panelists' statements and is reported in a later section of this summary.

This paper presents results of an empirical analysis of NATE (Network Analysis of Anomalous Traffic Events), a lightweight, anomaly based intrusion detection tool. Previous work was based on the simulated Lincoln Labs data set. Here, we show that NATE can operate under the constraints of real data inconsistencies. In addition, new TCP sampling and distance methods are presented. Differences between real and simulated data are discussed in the course of the analysis.

Using certificates for distributed authorizations in computer network systems has been discussed in the literature. However real implementations of the concept are rarely seen. In our certificate based authorization simulation system (CBASS) project, we prototyped a computer system including some of the emulated functions of an operating system such as machine, user and file management, and emulated applications. These system management functions and applications use certification instead of conventional access control list mechanism for resource access control. It is our purpose to explore a practical way to build the distributed authorization into computer systems in order to solve authorization problems which exist in present open and large distributed computing environments. In this paper, we present our design and implementation of the CBASS.

Connection of a local area network to the Internet brings with it a certain amount of risk. Once connected, local computer systems are subject to attacks by other users on the Internet. To maintain system integrity, system administrators need tools to accurately display the status of communication between local hosts and remote hosts. The paper describes an ongoing research and development project for Surveying Network Information Flow (SNIF). The SNIF tool provides system administrators and security officers an encapsulated view of network communication and information flow for a set of monitored subnetworks. The tool provides a real-time graphical interface, configurable monitoring, alarms and signals, connection statistics, and an extensible interface to permit links to external systems (LES). This tool is one of many in a suite of security tools being developed at the University of Idaho.>

A generalization of D. McCullough's (1987; 1988) restrictiveness model is given as the basis for providing security properties for distributed system designs. This generalization is mechanized for an event-based model of computer systems in the HOL (higher order logic) system to prove the composability of the model and several other properties about the model. A set of generalized classes of system components is developed and it is shown for which families of user views they satisfy the model. Using these classes, a collection of general system components that are specializations of one of these classes is delineated and it is shown that the specializations also satisfy the security property. A sample distributed secure system is presented along with an example of how the proposed mechanized verification system can be used to verify such designs.>