The knowledge engineering of belief networks is discussed. Several design issues that arose during the construction of two belief network-based systems, Pathfinder and ARCO1, are described. The issues of accuracy, consistency, and calibration as they emerged during the design of these systems are addressed, and the ways in which compatibility of all networks designed for the same domain suggests an architecture for combining the recommendations of independently designed knowledge bases into a single, consensus recommendation are discussed.>

Strategic planners for robots that survive in a realistic environment must coordinate sensor acquisition with robotic activity. Sensory coordination is discussed as a decision problem. It is assumed that although relevant sensory data can generally be requested, costs prohibit many of these requests from being granted. The underlying question is whether a central controller should grant or deny a system's request for a sensory update. Given this fundamental problem, a decision-theoretic analysis is used to derive closed-form formulas for the appropriate frequency of sensor integration as a function of parameters of the equipment, the domain, and the types of errors from which the system must recover. The derived formulas give precise descriptions of the frequency with which sensory requests should be granted in a simplified, formalized setting. They also serve as a template into which a variety of real-world parameterizations could fit.>

Consensus diagnoses arise when several experts contribute their opinions about the relative merits of a series of competing hypothesis, and a single decision maker combines their responses and makes a decision without further discussion among the contributors. Consensus diagnoses were simulated by allowing an oracle to generate 'opinions' based on universal background knowledge and all available information about the specific problem being diagnosed. Contributors' opinions were generated by perturbing the oracle's opinion; the size of the perturbation depended on the contributor's degree of expertise. Several different aggregation functions were then used to reclaim the oracle's opinion from those of the contributors. The performance of these functions was compared as panel size and hypothesis-set size varied from two to ten. Comparative and individual analyses indicated that for panels assembled under circumstances similar to those of this study, small, simple methods work best.>

Static evaluators have been used in every game program ever written. These heuristic functions attempt to differentiate between strong and weak moves by assigning them values based on directly detectable game features. Despite their ubiquity, evaluation functions are not well understood; the development of a theory of evaluator design has been too long coming. In fact, the general consensus is that no theory is possible, because expertise is required to develop even a simple evaluator. One recently introduced ‘generic’ evaluation function, the expected-outcome model, proposed evaluating a node as the expected value of a game's outcome, given random play from that node on. Experimental studies conducted on evaluators designed under this model yielded encouraging results in tac-tac-toe, Othello, and chess. This paper analyzes the expected-outcome model on a simple class of game trees, and shows that the moves recommended under the assumptions of random play and perfect play are identical. This vindicates what appeared to have been an overly naive assumption, and furthers the claim that statistically interpreted evaluation functions are powerful, as well as elegant.

. Artificial intelligence (AI) is. in many respects, an experimental science. Nevertheless, little has been written about the applicability of the scientific method to AI. The literature concerning techniques for learning and testing static evaluators provides a case in point: experiments are widely discussed, but rarely analyzed. These experiments have played prominent roles in both heuristic search and machine learning; in general, progressively stronger evaluators are learned and competitive abilities are tested. There are two basic problems with this approach. First, learning and improvement are not synonymous; there have been few attempts to specify precisely what is being learned. Second, competition blurs a program' s constituent components into a single performance measure. An alternative approach would be to identify a key parameter, to learn evaluators that approximate it, and then to challenge otherwise identical programs with marginally different functions. This paper reviews some previous work in respect to its adherence to the general principles of experiment design. It also describes experiments that learn coefficients for chess material-advantage functions that approximate expected outcome, and then describes a tournament involving expert-design, learned and randomly generated material-advantage evaluators. The results of this tournament challenge many long-held beliefs and stress the need for greater precision in experimental AI.

The expected-outcome model, in which the proper evaluation of a game-tree node is the expected value of the game's outcome given random play from that node on, is proposed. Expected outcome is considered in its ideal form, where it is shown to be a powerful heuristic. The ability of a simple random sampler that estimates expected outcome to outduel a standard Othello evaluator is demonstrated. The sampler is combined with a linear regression procedure to produce efficient expected-outcome estimators. Overall, the expected-outcome model of two-player games is shown to be precise, accurate, easily estimable, efficiently calculable, and domain-independent.>

A prescriptive approach to system design is outlined which breaks from standard expert systems research on two key points. First, the performance of a well-defined task analysis is taken to be a prerequisite for system design. It is the domain, then, and not the manner in which people cope with it, that guides the design phase. Second, an epistemological hierarchy, which recognizes expertise as a (high) range on a competence scale, helps identify the parts of a domain that are relevant to different types of users. This removes knowledge-based systems from the sole proprietorship of experts, and suggests using them as widely available advisors or decision support systems.>