This paper presents the findings of a survey of software tools built to assist in the verification and validation of knowledge-based systems. The tools were identified from literature sources from the period 1985–1995. The tool builders were contacted and asked to complete and return a survey that identified which testing and analysis techniques were utilised and covered by their tool. From these survey results it is possible to identify trends in tool development, technique coverage and areas for future research.

This paper describes a multilevel development life cycle of representation refinement for knowledge-based systems that incorporates meta-knowledge at each level. The methodology uses formal techniques in the specification of the domain knowledge, the cognitive aspects and the representation. The methodology provides the knowledge engineer with a dynamic perspective of the system which can be used in conjunction with the static aspects found in the representation abstractions. To provide perspective, the paper details the refinement of one of the levels called the intermediate level, in which an implementation-independent representation is created by the use of a knowledge filter.

This paper takes a parallel processing approach to the implementation of rule-based systems using a graph-reduction architecture, and investigates the consequences of this architecture in relation to the validation and verification of knowledge-based systems. The paper improves on the traditional sequential approaches to the development of knowledge-based systems and the limited validation and verification techniques that are applicable. This is contrasted with a graph reduction implementation of knowledge-based systems development based on an ALICE-like machine. The advantages of this style of programming in relation to systems development and program correctness are discussed. The paper shows that significant benefits could potentially be achieved through the use of graph-reduction techniques in the development of these systems.

This paper integrates the premise that current software level practices within the aerospace industry are weak and that there is a lack of rigour in both technical and managerial areas. Results from a survey of practitioners are presented that indicate a lack of information interchange exists and that the use of formal techniques is limited. The paper proposes that this is indicative of poor life-cycle practices and that more rigorous methodologies, ones that integrate formal methods with quality practices, are required. A two-level model is proposed to address the issue.

The existence of a formal description of any language is a prerequisite to any rigorous methods of proof, validation, or verification. This article develops a formal semantics in the denotational style for rule-based production systems. Primarily, a formal description of a generalized, hypothetical language for rule-based expert systems is developed as a base. Building on this base, it is possible to specify in detail the variations that distinguish individual real-world systems in a usable way.

The term quality has been interpreted in a variety of ways, including definitions by researchers such Garvin (1984), who set out his 8 dimensions quality as follows:

The paper explores the implementation of rule-based pattern-directed inference systems on parallel computers. The paper discusses one of these approaches in detail, the use of a graph-reduction machine such as ALICE. The technique is illustrated through two example domains: automobile fault diagnosis and organic psychiatric mental disorders. The paper discusses extensions to the graph reduction technique as applied to knowledge-based systems, including partitioning, time considerations and input data types. The paper shows that the graph-reduction technique has significant advantages for knowledge-based system implementation over conventional approaches, and it demonstrates that this programming style is amenable to knowledge engineering domains.

Abstract The decision table is one of the simplest representations of the rules underlying a systematic decision‐making process, and is especially valuable in the development of knowledge‐based systems. A two dimensional table links all relevant combinations of input conditions to the desired combinations of output actions in a very intuitive way. This simplicity belies the complex considerations involved in verifying, validating, formulating or interpreting this (or any other) representation of machine‐based knowledge. In this paper, the common styles of decision table representation are reviewed, a formulation of their meaning is presented, construction methods are reviewed, and an algorithm for ensuring consistency is suggested. The problems that may occur in imperfectly constructed tables are discussed, detection methods reviewed, and some implementation methods are presented.

The article presents a formal specification for many important aspects of the OPS5 production systems framework. the article illustrates how an abstract formal specification of a production system can be created and the benefits this provides to those involved in the development of knowledge-based systems. the formal specification is preceded by an informal specification of a production system upon which the formal model is based and the development is illustrated through the use of concrete examples. the notation used is that of “Z” (J. M. Spivey, The Z Notation, Prentice-Hall, Englewood Cliffs, NJ, 1990), a language based upon typed set theory. This language has been used to success in the specification of critical conventional software systems (I. Hayes, Technical Monograph PRG-46, Oxford University Computing Laboratory, Oxford, England, 1985) and which is formal enough to allow for the creation of rigorous specifications, yet is of a form that makes these specifications “readable.” the aim of the article is to show that formal techniques can be applied to areas of knowledge-based system development, thus promoting correctness, reliability, and understanding. © 1994 John Wiley & Sons, Inc.

Abstract This article discusses the problems found in the validation and verification of a knowledge-based system for equity selection. These problems include the selection of test data, poor methodology, and the difficulties associated with using prototypes. The article then examines the possible techniques available to the knowledge engineer for improving validation and verification. The article discusses exhaustive testing, case-based testing, formal specifications, functional programming, critical testing, mutation testing, and reliability. Finally the article discusses the approach that the knowledge engineer would take in rewriting the equity selection system, one based on a rigorous development methodology that uses as many formal validation techniques as possible to raise the quality of the software produced.