Inductive machine learning has become an important approach to automated knowledge acquisition from databases. The disjunctive normal form (DNF), as the common analytic representation of decision trees and decision tables (rules), provides a basis for formal analysis of uncertainty and complexity in inductive learning. A theory for general decision trees is developed based on C. Shannon's (1949) expansion of the discrete DNF, and a probabilistic induction system PIK is further developed for extracting knowledge from real world data. Then we combine formal and practical approaches to study how data characteristics affect the uncertainty and complexity in inductive learning. Three important data characteristics, namely, disjunctiveness, noise and incompleteness, are studied. The combination of leveled pruning, leveled condensing and resampling estimation turns out to be a very powerful method for dealing with highly disjunctive and inadequate data. Finally the PIK system is compared with other recent inductive learning systems on a number of real world domains.>

The authors develop a theory for general decision tree induction based on both the logical structure of concepts and the probability distribution of examples. The discrete function is the common analytic representation of decision trees and decision tables (rules). One of the most important classes of discrete functions is the disjunctive normal forms (DNF). Disjunctiveness of concepts has a great effect on the accuracy and speed of concept learning. A theory for general decision trees is developed based on Shannon's expansion of the discrete DNF. The function-equivalence, the structural manipulations, and irreducible DNFs and trees are studied. For optimizing decision trees in the context of induction, the functional and structural criteria are investigated.

The authors present a statistical-heuristic feature selection criterion for constructing multibranching decision trees in noisy real-world domains. Real world problems often have multivalued features. To these problems, multibranching decision trees provide a more efficient and more comprehensible solution that binary decision trees. The authors propose a statistical-heuristic criterion, the symmetrical tau and then discuss its consistency with a Bayesian classifier and its built-in statistical test. The combination of a measure of proportional-reduction-in-error and cost-of-complexity heuristic enables the symmetrical tau to be a powerful criterion with many merits, including robustness to noise, fairness to multivalued features, and ability to handle a Boolean combination of logical features, and middle-cut preference. The tau criterion also provides a natural basis for prepruning and dynamic error estimation. Illustrative examples are also presented.>