Suppose that, for purposes of inductive inference or choosing an optimal decision, we wish to select a single distribution P* to act as representative of a class /spl Gamma/ of such distributions. The maximum entropy principle ("MaxeEnt") (Jaynes 1989; Csiszar 1991) is widely applied for this purpose, but its rationale has often been controversial (Shimony 1985; Seidenfeld 1986). Here we emphasize and generalize a reinterpretation of the maximum entropy principle (Topsoe (1979); Walley (1991); Grunwald (1998)): that the distribution P* that maximizes the entropy over /spl Gamma/ also minimizes the worst-case expected logarithmic score (log loss). In the terminology of decision theory (Berger 1985), P* is a robust Bayes, or /spl Gamma/-minimax, act, when loss is measured by the log loss. This gives a decision-theoretic justification for maximum entropy.

It is 12 years since I opened the discussion at the Royal Statistical Society meeting at which Rissanen and Wallace and Freeman presented companion papers on topics not far removed from those under discussion now. In the interim our understanding of the relationship between statistical inference, coding theory and algorithmic complexity has developed somewhat, but I feel there are still gaps to bridge. Rather than comment in detail on the current papers, I propose to use this opportunity to attempt a review of the whole area of stochastic complexity, or MDL, from a personal perspective.

Probabilistic expert systems based on Bayesian networks require initial specification of both qualitative graphical structure and quantitative conditional probability assessments. As (possibly incomplete) data accumulate on real cases, the parameters of the system may adapt, but it is also essential that the initial specifications be monitored with respect to their predictive performance. A range of monitors based on standardized scoring rules that are designed to detect both qualitative and quantitative departures from the specified model is presented. A simulation study demonstrates the efficacy of these monitors at uncovering such departures.>

Abstract We investigate directed Markov fields over finite graphs without positivity assumptions on the densities involved. A criterion for conditional independence of two groups of variables given a third is given and named as the directed, global Markov property. We give a simple proof of the fact that the directed, local Markov property and directed, global Markov property are equivalent and – in the case of absolute continuity w. r. t. a product measure – equivalent to the recursive factorization of densities. It is argued that our criterion is easy to use, it is sharper than that given by Kiiveri, Speed, and Carlin and equivalent to that of Pearl. It follows that our criterion cannot be sharpened.