In this paper we describe a technique for generating expected results for automated black-box testing. Generating expected results allows larger automated test suites to be created, moving us toward continuous product testing. Our technique uses a program's Input-Output (IO) relationships to identify unique combinations of program inputs that influence program outputs. With this information, a small set of test cases is executed and checked for correctness. Given the correctness of this set, the expected results for the larger combinatorial test set can be generated automatically. Included in the paper is an experimental study in which checking the results of 384 test cases allows us to generate expected results and fully automate nearly 600,000 test cases.

Test reduction is an important issue in black-box testing. The number of possible black-box tests for any non-trivial software application is extremely large. For the class of programs with multiple inputs and outputs, the number of possible tests grows very rapidly as combinations of input test data are considered. In this paper, we introduce an approach to test reduction that uses automated input-output analysis to identify relationships between program inputs and outputs. Our initial experience with the approach has shown that it can significantly reduce the number of black-box tests.