Errors can arise in defining and evaluating computer security policy as well as in translating computer security policy into procedures.The effect of such errors in policy upon the secure operation of information systems can impose unacceptable levels of risk from the perspective of procurers and users of information systems.Relying on computer security paradigms based solely on formal methods makes it difficult if not impossible to detect and/or reason about certain classes of threats to computer security and vulnerabilities of information systems to these threats, especially for those aspects of information systems that are more readily amenable to modeling via non-formal methods.We present a paradigm integrating formal and heuristic reasoning as a basis for testing for and debugging computer security policy.To illustrate our approach, and to support our arguments, we consider the problem of reasoning about the plans of an agent who may be trying to compromise the security of an information system.

Three significant challenges facing the discipline of software engineering are understanding how software designers reason about policy, understanding how they define requirements based on this reasoning, and providing machine-based intelligent tools that support reasoning about policy. The authors are developing the concept of a policy workbench: knowledge-based tools that support reasoning about such properties of policies as consistency, completeness, and correctness, and the implications of policy for the behavior of a system. They discuss how such a workbench might help software designers reason about the relationships between policies and requirements.>

This paper reports the results of an empirical analysis of the knowledge engineering behavior of six persons with extensive experience in artificial intelligence (AI). The six persons were given the task of designing an AI program and were videotaped while they did so; during their 2–3 h design sessions, they were asked to talk aloud about what they were doing and why they were doing it. The paper identifies several recurrent behaviors common to all the AI designers. For example, several components of the designers' goal structures are identified, as is the importance of focussing on a “touchstone”, or key issue, around which much of the designer's behavior revolves. Several potential implications of the research for the design of knowledge engineering tools are explored.