Theory revision integrates inductive learning and background knowledge by combining training examples with a coarse domain theory to produce a more accurate theory. There are two challenges that theory revision and other theory-guided systems face. First, a representation language appropriate for the initial theory may be inappropriate for an improved theory. While the original representation may concisely express the initial theory, a more accurate theory forced to use that same representation may be bulky, cumbersome, and difficult to reach. Second, a theory structure suitable for a coarse domain theory may be insufficient for a fine-tuned theory. Systems that produce only small, local changes to a theory have limited value for accomplishing complex structural alterations that may be required. Consequently, advanced theory-guided learning systems require flexible representation and flexible structure. An analysis of various theory revision systems and theory-guided learning systems reveals specific strengths and weaknesses in terms of these two desired properties. Designed to capture the underlying qualities of each system, a new system uses theory-guided constructive induction. Experiments in three domains show improvement over previous theory-guided systems. This leads to a study of the behavior, limitations, and potential of theory-guided constructive induction.

A new neural network called AUGURS is designed to assist a user of a computer-aided design package in utilizing standard graphical symbols. AUGURS is similar to the Zipcode Net by Le Cun et al. (1989, 1990) in its encoding of transformation knowledge into its network structure, but is much more compact and efficient. The experiments compare AUGURS with two versions of the Zipcode Net and a traditional layered feedforward network with an unconstrained structure. The experimental results show that AUGURS can recognize a user-drawn symbol with better accuracy and plausibility than the other networks with the least amount of recognition time when the number of training examples is limited.

A knowledge base and learning system is designed to help biologists predict protein structure and function based on sequence information. The knowledge base contains diverse information about: (a) protein motif sequences and structures, (b) heuristics and programs for identifying protein features, and (c) molecular simulation programs. The learning system selects the most relevant information for a particular prediction task and optimally integrates the information to generate accurate and comprehensible hypotheses. Biologists define the objectives for learning such as accuracy and comprehensibility. To overcome the limitations of existing induction algorithms, techniques are developed for constructing new features based on the existing knowledge base. Optimization algorithms are used for determining the best combination of induction and feature construction strategies for a problem.>

The intrinsic accuracy of an inductive problem is the accuracy achieved by exhaustive table look‐up. Intrinsic accuracy is the upper bound for any inductive method. Hard concepts are concepts that have high intrinsic accuracy, but which cannot be learned effectively with traditional inductive methods. To learn hard concepts, we must use constructive induction ‐ methods that create new features. We use measures of concept dispersion to explore (conceptually and empirically) the inherent weaknesses of traditional inductive approaches. These structural defects are buried in the design of the algorithms and prevent the learning of hard concepts. After studying some examples of successful and unsuccessful feature construction (“success” being defined here in terms of accuracy), we introduce a single measure of inductive difficulty that we call variation. We argue for a specific approach to constructive induction that reduces variation by incorporating various kinds of domain knowledge. All of these kinds of domain knowledge boil down to utility invariants, i.e., transformations that group together non‐contiguous portions of feature space having similar class‐membership values. Utility invariants manifest themselves in various ways: in some cases they exist in the user's stock of domain knowledge, in other cases they may be discovered via methods we describe.

A representative (but unimodal) problem, the fifteen puzzle, is used to test six different learning systems: some that fit, count, or partition data in instance space; some that optimize measures derived from data in hypothesis space; and some that perform combinations of such procedures. The six systems are described, tested, and analyzed. Through several experiments, specific properties are extracted. By combining two or three kinds of techniques, the extent to which they complement each other is gauged. It is concluded that combinations of strengths can overcome difficulties in domains that are simultaneously probabilistic, dynamic, noisy, and biased.>

This paper describes a novel approach to machine learning, based on the principle of learning by reasoning. Current learning systems have significant limitations such as brittleness, i.e., the deterioration of performance on a different domain or problem and lack of power required for handling real‐world learning problems. The goal of our research was to develop an approach in which many of these limitations are overcome in a unified, coherent and general framework. Our learning approach is based on principles of reasoning, such as the discovery of the underlying principle and the recognition of the deeper basis of similarity, which is somewhat akin to human learning. In this paper, we argue the importance of these principles and tie the limitations of current systems to the lack of application of these principles. We then present the technique developed and illustrate it on a learning problem not directly solvable by previous approaches.

This paper synthesizes a number of approaches to concept representation and learning in a multilayered model. The paper emphasizes what has been called similarity‐based learning (SBL) from examples, although this review is extended to address wider issues. The paper pays particular attention to requirements for incremental and uncertain environments, and to interrelationships among concept purpose, concept representation, and concept learning. One goal of the paper is to unite some of the notions underlying recent research, in an attempt to construct a more complete and extensible framework. This framework is designed to capture representations and methods such as those based on hypothesis search and bias selection, and to extend the ideas for greater system capability. This leads to a specific perspective for multilayered learning which has several advantages, such as greater clarity, more uniform learning, and more powerful induction. The approach clarifies and unifies various aspects of the problem of concept learning. Some results'are (1) Various concept representations (such as logic, prototypes, and decision trees) are subsumed by a standard form which is well suited to learning, particularly in incremental and uncertain environments; (2) Concept learning may be enhanced by exploiting a particular phenomenon in many spaces‐this phenomenon is a certain kind of smoothness or regularity, one instance of which underlies the similarity in SBL systems; (3) The paper treats the phenomenon in a general way and applies it hierarchically. This has various advantages of uniformity. For example the model allows layered learning algorithms for concept learning all to be instantiations of one basic algorithm. A single kind of representation (an instantiation of the standard form) is prominent at each level. The combination of representation and algorithm allows fast, accurate, concise, and robust concept learning.