This paper briefly describes the AQ21 learning system that implements a simple form of natural induction, an approach to learning that generates hypotheses in forms resembling natural language descriptions, and by that easy to understand and interpret. The system was applied to the analysis of aggregated data obtained from non-invasive tests performed on different groups of patients with metabolic syndrome. The discovered patterns were very simple and were evaluated by an expert as potentially medically significant.

Learnable Evolution Model (LEM) is a form of non-Darwinian evolutionary computation that employs machine learning to guide evolutionary processes. Its main novelty are new type of operators for creating new individuals, specifically, hypothesis generation, which learns rules indicating subareas in the search space that likely contain the optimum, and hypothesis instantiation, which populates these subspaces with new individuals. This paper briefly describes the newest and most advanced implementation of learnable evolution, LEM3, its novel features, and results from its comparison with a conventional, Darwinian-type evolutionary computation program (EA), a cultural evolution algorithm (CA), and the estimation of distribution algorithm (EDA) on selected function optimization problems (with the number of variables varying up to 1000). In every experiment, LEM3 outperformed the compared programs in terms of the evolution length (the number of fitness evaluations needed to achieved a desired solution), sometimes more than by one order of magnitude.

A new method for optimizing complex functions and systems is described that employs learnable evolution model (LEM), a form of non-Darwinian evolutionary computation guided by machine learning. LEM's main novelties are operators for creating new individuals that include hypothesis generation, which learns rules indicating subareas in the search space likely containing the optimum, and hypothesis instantiation, which populates these subareas with new candidate solutions. LEM3, the newest and most advanced implementation of learnable evolution, is briefly described and experimentally compared with other evolutionary computation programs on selected function optimization problems. We also describe two specialized LEM-based systems for heat exchanger optimization

The AQ21 program aims to perform natural induction, a process of generating inductive hypotheses in human-oriented forms that are easy to interpret and understand. This is achieved by employing a highly expressive representation language, attributional calculus, whose statements resemble natural language descriptions. This paper focuses on the pattern discovery mode of AQ21, which produces attributional rules that capture strong regularities in the data, but may not be fully consistent or complete with regard to the training data. AQ21 integrates several novel features, such as optimizing patterns according to multiple criteria, learning attributional rules with exceptions, generating optimized sets of alternative hypotheses, and handling data with unknown, irrelevant and/or non-applicable meta-values

A new method for optimizing complex engineering designs is presented that is based on the Learnable Evolution Model (LEM), a recently developed form of non-Darwinian evolutionary computation. Unlike conventional Darwinian-type methods that execute an unguided evolutionary process, the proposed method, called LEMd, guides the evolutionary design process using a combination of two methods, one involving computational intelligence and the other involving encoded expert knowledge. Specifically, LEMd integrates two modes of operation, Learning Mode and Probing Mode. Learning Mode applies a machine learning program to create new designs through hypothesis generation and instantiation, whereas Probing Mode creates them by applying expert-suggested design modification operators tailored to the specific design problem. The LEMd method has been used to implement two initial systems, ISHED1 and ISCOD1, specialized for the optimization of evaporators and condensers in cooling systems, respectively. The designs produced by these systems matched or exceeded in performance the best designs developed by human experts. These promising results and the generality of the presented method suggest that LEMd offers a powerful new tool for optimizing complex engineering systems. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 1217–1248, 2006.

A recently developed approach to evolutionary computation, called Learnable Evolution Model or LEM, employs machine learning to guide processes of generating new populations. The central new idea of LEM is that it generates new individuals not by mutation and/or recombination, but by processes of hypothesis generation and instantiation. The hypotheses are generated by a machine learning system from examples of high and low performance individuals. When applied to problems of function optimization and parameter estimation for nonlinear filters, LEM significantly outperformed the standard evolutionary computation algorithms used in experiments, sometimes achieving two or more orders of magnitude of evolutionary speed-up (in terms of the number of births). An application of LEM to the problem of optimizing heat exchangers has produced designs equal to or exceeding the best human designs. Further research needs to explore trade-offs and determine best areas for LEM application.

Knowledge scouts are software agents that autonomously synthesize knowledge of interest to a given user (target knowledge) by applying inductive database operators to a local or distributed dataset. This paper describes briefly a method and a scripting language for developing knowledge scouts, and then reports on experiments with a knowledge scout, SCAMP, for discovering patterns characterizing relationships among lifestyles, symptoms and diseases in a large medical database. Discovered patterns are presented in two forms: (1) attributional rules, which are expressions in attributional calculus, and (2) association graphs, which graphically and abstractly represent relations expressed by the rules. Preliminary results indicate a high potential utility of the presented methodology for deriving useful and understandable knowledge. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Knowledge scouts are software agents that autonomously synthesize user-oriented knowledge (target knowledge) from information present in local or distributed databases. A knowledge generation metalanguage, KGL, is used to creating scripts defining such knowledge scouts. Knowledge scouts operate in an inductive database, by which we mean a database system in which conventional data and knowledge management operators are integrated with a wide range of data mining and inductive inference operators. Discovered knowledge is represented in two forms: (1) attributional rules, which are rules in attributional calculus—a logic-based language between prepositional and predicate calculus, and (2) association graphs, which graphically and abstractly represent relations expressed by the rules. These graphs can depict multi-argument relationships among different concepts, with a visual indication of the relative strength of each dependency. Presented ideas are illustrated by two simple knowledge scouts, one that seeks relations among lifestyles, environmental conditions, symptoms and diseases in a large medical database, and another that searches for patterns of children's behavior in the National Youth Survey database. The preliminary results indicate a high potential utility of this methodology for deriving knowledge from databases.

An enormous proliferation of computer technology in modern societies has produced a severe information overload. The navigation through the masses of available information in order to derive desired knowledge is becoming increasingly difficult. This

A decision structure is a simple and powerful tool for organizing a decision process. It differs from a conventional decision tree in that its nodes are assigned tests that can be functions of the attributes, rather than single attributes; the branches stemming from a node can be assigned a subset of attribute values rather than a single attribute value (test outcome); and the leaves can be assigned one or more alternative decisions. We describe a methodology for learning decision structures from declarative knowledge expressed in the form of decision rules. The decision rules are generated by an expert, or by an AQ-type inductive learning program (with or without constructive induction). From a given set of rules, one can generate many different decision structures. The proposed methodology generates the one that is most suitable for the given decision-making situation, according to a multicriterion evaluation function. Experiments with a program implementing the proposed methodology have demonstrated its many useful features.

This paper describes a partial-memory incremental learning method based on the AQ15c inductive learning system. The method maintains a representative set of past training examples that are used together with new examples to appropriately modify the currently held hypotheses. Incremental learning is evoked by feedback from the environment or from the user. Such a method is useful in applications involving intelligent agents acting in a changing environment, active vision, and dynamic knowledge-bases. For this study, the method is applied to the problem of computer intrusion detection in which symbolic profiles are learned for a computer system's users. In the experiments, the proposed method yielded significant gains in terms of learning time and memory requirements at the expense of slightly lower predictive accuracy and higher concept complexity, when compared to batch learning, in which all examples are given at once.

The TEXTRAL system, used for determining structural visual properties of textures through symbolic transformations, is presented. The method consists of two phases: one that extracts information from raw textural images by applying convolutional operators and learns an initial set of rules; and a second that iteratively extracts symbolic information from the transformed representation of initial image and learns another set of rules. The transformed symbolic representation is obtained by applying previously learned rules to a new image location and generating symbolic images based on rule assertions.>

A method is presented for constructive induction, in which new attributes are constructed as various functions of original attributes. Such a method is called data-driven constructive induction, because new attributes are derived from an analysis of the data (examples) rather than the generated rules. Attribute construction and rule generation are repeated until a termination condition, such as the satisfaction of a rule quality measure, is met. The first step of this method, the generation of new attributes, has been implemented in AQ17-PRE. Initial experiments with AQ17-PRE have shown that it leads to an improvement of the learned rules in terms of both their simplicity and their accuracy on testing examples.>

Machine learning, a field concerned with developing computational theories of learning and constructing learning machines, is now one of the most active research areas in artificial intelligence. An inference-based theory of learning will be presented that unifies basic learning strategies. Special attention will be given to comparing and unifying inductive learning and deductive learning strategies.Inductive learning strategies include empirical techniques for learning from examples and learning from observation and discovery. Deductive learning techniques include analytic learning on the basis of the explanation of a given fact using prior domain knowledge. We will show that the “similarity-based learning” (a form of inductive learning) and the “explanation-based learning” (a form of deductive learning) are two extremes in the spectrum of techniques representing different relative role of the learner's prior knowledge and the information supplied to the learner. We will also show how inductive and deductive learning can be integrated within one theoretical framework. Some experimental results will be used to illustrate presented ideas.

A method for automated construction of classifications called conceptual clustering is described and compared to methods used in numerical taxonomy. This method arranges objects into classes representing certain descriptive concepts, rather than into classes defined solely by a similarity metric in some a priori defined attribute space. A specific form of the method is conjunctive conceptual clustering, in which descriptive concepts are conjunctive statements involving relations on selected object attributes and optimized according to an assumed global criterion of clustering quality. The method, implemented in program CLUSTER/2, is tested together with 18 numerical taxonomy methods on two exemplary problems: 1) a construction of a classification of popular microcomputers and 2) the reconstruction of a classification of selected plant disease categories. In both experiments, the majority of numerical taxonomy methods (14 out of 18) produced results which were difficult to interpret and seemed to be arbitrary. In contrast to this, the conceptual clustering method produced results that had a simple interpretation and corresponded well to solutions preferred by people.

The determination of pattern recognition rules is viewed as a problem of inductive inference, guided by generalization rules, which control the generalization process, and problem knowledge rules, which represent the underlying semantics relevant to the recognition problem under consideration. The paper formulates the theoretical framework and a method for inferring general and optimal (according to certain criteria) descriptions of object classes from examples of classification or partial descriptions. The language for expressing the class descriptions and the guidance rules is an extension of the first-order predicate calculus, called variable-valued logic calculus VL21. VL21 involves typed variables and contains several new operators especially suited for conducting inductive inference, such as selector, internal disjunction, internal conjunction, exception, and generalization. Important aspects of the theory include: 1) a formulation of several kinds of generalization rules; 2) an ability to uniformly and adequately handle descriptors (i.e., variables, functions, and predicates) of different type (nominal, linear, and structured) and of different arity (i.e., different number of arguments); 3) an ability to generate new descriptors, which are derived from the initial descriptors through a rule-based system (i.e., an ability to conduct the so called constructive induction); 4) an ability to use the semantics underlying the problem under consideration. An experimental computer implementation of the method is briefly described and illustrated by an example.