A production system (PS) is a forward chaining rule based system used to build large expert systems. Testing a PS must involve the construction of a covering set of test data but it is not clear what the meaning of covering a PS is and how a test data set can be measured according to coverage. We propose a test data coverage measure for a subset for PS with well defined semantics. We use a correspondence between PS and function free first order Horn logic programs to define the declarative coverage notion and measure. We found that the coverage measure can be used to determine the coverage of the program logic of a PS as well as to automate test data generation. Unification theory is utilised to measure test data coverage and constrained inductive generation is used for test data construction.

Two facts about declarative programming prevent the application of conventional testing methods. First, the classical test coverage measures such as statement, branch or path coverage, cannot be used, since in declarative programs no control flow notion exists. Second, there is no widely accepted language available for formal specification, since predicate logic, which is the most common formalism for declarative programming, is already a very high-level abstract language. This paper presents a new approach exending previous work by the authors on test input generation for declarative programs. For this purpose, the existing program instrumentation notion is extended and a new logic coverage measure is introduced. The approach is mathematically formalized and the goal of achieving 100% program logic coverage controls the automatic test input generation. The method is depicted by means of logic programming; the results are, however, generally applicable. Finally, the concepts introduced have been used practically within a test environment. © 1998 John Wiley & Sons, Ltd.

Reliability determination for software is closely related to software testing. Testing delivers important data for software reliability models. Two important tasks of software testing are test case generation and determination of test coverage. Because of its declarative paradigm, all the well-known approaches to the above mentioned tasks are not applicable to logic programming. Implementation based testing is an approach to tackle the test problem for logic programming. We focus on test coverage aspects for logic programming. Analog to the coverage notion for conventional programming, e.g., path coverage on the control flow graph, we define a coverage measure on an abstract model of the logic program to be tested. Our abstract model is the set of goal-induced instances of program clauses. This is motivated by the computational model of resolution calculus for first-order predicate logic. Anti-unification is utilized to define the coverage measure via least upper bounds of this set. We give an algorithm for test cover determination which is prepared for coverage oriented test input generation, e.g., by declarative program instrumentation or non goal-oriented execution of logic programs using a fair interpreter. We describe syntactic and semantic aspects of our testing approach with respect to the properties of logic programming.

The systematic approaches to testing and reliability determination of programs e.g. are applicable to imperative programming but not immediately to declarative programming, such as logic programming, which is of great importance to develop knowledge-based systems. We describe an approach to implementation-based testing and reliability determination of logic programs materialized in a product assurance environment, presently limited to two major components: the test environment PROTest and the reliability assessment environment PRORool, with the results of the former serving as input for the latter. The test environment consists of structure analysis of logic programs, automatic test case generation and execution, test coverage determination, and generation of test reports. The reliability assessment environment provides an approach to reliability prediction and estimation of Prolog programs, introducing two measures describing Prolog programs complexity, which are used to determine the program reliability. It implements also several well-known software reliability models for comparison purposes.

In this paper, we describe the PROTest II (PROlog Test Environment, Version 2) system to test logic programs in an interactive support environment. Logic programs are augmented with declarative information about the types and modes of the arguments of a predicate. Modes correspond to in, out, and in-out parameters. With this information PROTest II statically checks the types of Prolog programs, generates test cases, executes Prolog programs, and produces reports summarizing results including information about new test coverage metrics. Thus, PROTest II enables both static analysis and dynamic testing uniformly using a Prolog-based test language DTL/1. The strength of PROTest II stems from its idea of defining coverage in real logic programming terms, rather than adapting imperative programming ideas.