Rules are commonly used for classification because they are modular intelligible and easy to learn. Existing work in classification rule learning assumes the goal is to produce categorical classifications to maximize classification accuracy. Recent work in machine learning has pointed out the limitations of classification accuracy: when class distributions are skewed or error costs are unequal, an accuracy-maximizing rule set can perform poorly. A more flexible use of a rule set is to produce instance scores indicating the likelihood that an instance belongs to a given class. With such an ability, we can apply rule sets effectively when distributions are skewed or error costs are unequal. This paper empirically investigates different strategies for evaluating rule sets when the goal is to maximize the scoring (ROC) performance.

Since Samuel's work on checkers over thirty years ago, much effort has been devoted to learning evaluation functions. However, all such methods are sensitive to the feature set chosen to represent the examples. If the features do not capture aspects of the examples significant for problem solving, the learned evaluation function may be inaccurate or inconsistent. Typically, good feature sets are carefully handcrafted and a great deal of time and effort goes into refining and tuning them. This paper presents an automatic knowledge‐based method for generating features for evaluation functions. The feature set is developed iteratively: features are generated, then evaluated, and this information is used to develop new features in turn. Both the contribution of a feature and its computational expense are considered in determining whether and how to develop it further. This method has been applied to two problem‐solving domains: the Othello board game and the domain of telecommunications network management. Empirical results show that the method is able to generate many known features and several novel features and to improve concept accuracy in both domains.

Intelligent and Cooperative Information Systems (ICIS) will have large numbers of distributed, heterogeneous agents interacting and cooperating to solve problems regardless of location, original mission, or platform. The agents in an ICIS will adapt to new and possibly surprising situations, preferably without human intervention. These systems will not only control a domain, but also will improve their own performance over time, that is, they will learn. This paper describes five heterogeneous learning agents and how they are integrated into an Integrated Learning System (ILS) where some of the agents cooperate to improve performance. The issues involve coordinating distributed, cooperating, heterogeneous problem-solvers, combining various learning paradigms, and integrating different reasoning techniques. ILS also includes a central controller, called The Learning Coordinator (TLC), that manages the control of flow and communication among the agents, using a high-level communication protocol. In order to demonstrate the generality of the ILS architecture, we implemented an application which, through its own experience, learns how to control the traffic in a telephone network, and show the results for one set of experiments. Options for enhancements of the ILS architecture are also discussed.