Classical statistical inference is nonmonotonic: obtaining more evidence or obtaining more knowledge about the evidence one has can lead to the replacement of one statistical conclusion by another, or the complete withdrawal of the original conclusion. While it has long been argued that not all nonmonotonic inference can be accounted for in terms of relative frequencies or objective probabilities, there is no doubt that much nonmonotonic inference can be accounted for in this way. Here we seek to explore the close connection between classical statistical inference and default logic, treating statistical inference within the framework of default logic, and showing that nonmonotonic logic in general, and default logic in particular, needs to take account of certain features of statistical inference. Default logic must take account of statistics, but at the same time statistics can throw light on problematic cases of default inference.

In this paper, we focus on statistics. Classical statistics and Bayesian statistics are both employed in data mining. Both have advantages but both also have severe limitations in this context. We point out some of these limitations as well as some of the advantages. The fact that we may need to take account of evidence both internal and external to the data set presents a difficulty for classical statistics. The need to incorporate an objective measure of reliability creates a difficulty for Bayesian statistics. We outline an approach to uncertainty that promises to capture the best of both worlds by incorporating both background knowledge and objectivity.

Classical statistical inference is nonmonotonic in nature. We show how it can be formalized in the default logic framework. The structure of statistical inference is the same as that represented by default rules. In particular, the prerequisite corresponds to the sample statistics, the justifications require that we do not have any reason to believe that the sample is misleading, and the consequence corresponds to the conclusion sanctioned by the statistical test.

In ordinary first–order logic, a valid inference in a language L is one in which the conclusion is true in every model of the language in which the premises are true. To accommodate inductive/uncertain/probabilistic/nonmonotonic inference, we weaken that demand to the demand that the conclusion be true in a large proportion of the models in which the relevant premises are true. More generally, we say that an inference is [p,q] valid if its conclusion is true in a proportion lying between p and q of those models in which the relevant premises are true. If we include a statistical variable binding operator “%” in our language, there are many quite general (and useful) things we can say about uncertain validity. A surprising result is that some of these things may conflict with Bayesian conditionalization.

Problems for strict and convex Bayesianism are discussed. A set-based Bayesianism generalizing convex Bayesianism and intervalism is proposed. This approach abandons not only the strict Bayesian requirement of a unique real-valued probability function in any decision-making context but also the requirement of convexity for a set-based representation of uncertainty. Levi's E-admissibility decision criterion is retained and is shown to be applicable in the nonconvex case.

Introduction Do you believe that the temperature is between 64F and 66F when your well calibrated thermometer reads 65.1F? Do you believe that it will rain tomorrow when the newspaper forecasts rain? Many of us would reply affirmatively to the first question and negatively to the second, claiming that on reflection what we believe about the weather is that the chance of rain is 'high'. The human lust for uniformity and generality, however, has led others to seek a single answer. The cautious philosophical probabilist might point to the theory of error according to which errors are distributed approximately normally to conclude that relative to the evidence, the probability is merely extremely high that the temperature is between 64F and 66 F. The pragmatist might argue that just as taking the temperature to be between 64F and 66F provides a perfectly adequate basis for deciding what clothes to wear, so having the paper forecast rain provides a perfectly adequate basis for postp

Many researchers have felt uncomfortable with the precision of degrees of belief that seems to be demanded by the subjective Bayesian treatment of uncertainty. Various responses have been suggested. The most common one has been to incorporate higher order probabilities in systems that reason in beliefs. These probabilities concern statements of first-order probability. Thus a first-order probability (e.g., the probability of heads on the next toss of this coin is 12) is the subject of a second-order probability; for example, the probability is .9 that the probability of heads on the next toss of this coin is 12. This approach is explored and is found to be epistemologically wanting, although there are important intuitions about beliefs that are captured by it. Furthermore, this approach may, in some circumstances, be computationally attractive. We also briefly explore a number of other approaches, including taking probabilities to be intervals and construing probability values as fuzzy sets.

Four main results are arrived at in this paper. (1) Closed convex sets of classical probability functions provide a representation of belief that includes the representations provided by Shafer probability mass functions as a special case. (2) The impact of “uncertain evidence” can be (formally) represented by Dempster conditioning, in Shafer's framework. (3) The impact of “uncertain evidence” can be (formally) represented in the framework of convex sets of classical probabilities by classical conditionalization. (4) The probability intervals that result from Dempster-Shafer updating on uncertain evidence are included in (and may be properly included in) the intervals that result from Bayesian updating on uncertain evidence.