The decision of when to release a software product commercially is not a question of when the software has attained some objectively justifiable degree of correctness. It is, rather, a question of whether the software achieves a reasonable balance among engineering objectives, market demand, customer requirements, and marketing directives of the software organization. We present a rigorous framework for addressing this important decision. Conjugate distributions from statistical decision theory provide an attractive means of modeling the cost and rate of bugs given information acquired during software testing, as well as prior information provided by software engineers about the fidelity of the software before testing begins. In contrast to other methods, the stopping analysis yields a computationally simple rule for deciding when to release a commercial software product based on information revealed to engineers during software testing-complicated numerical procedures are not needed. Our method has the added benefits that it is sequential: it measures explicitly the costs of customer dissatisfaction associated with bugs as well as the costs of declining market position while the testing process continues; and it incorporates a practical framework for cost-criticality assessment that makes sense to professional software developers.

Because subjective probability assessments do not represent measurements of empirical quantities, they often exhibit vagueness, or ambiguity. We adopt a standard approach to modeling vagueness as a second-order probability (SOP) on a first-order probability. We define the expected value of including vagueness (EVIV) as the change in expected value due to explicit probabilistic representation of vagueness. We show that, under general conditions, the EVIV is always zero, regardless of the SOP or the loss or utility function. This shows that representing higher-order uncertainty as SOP never changes the output of a decision model. We analyze the value of reducing vagueness, and we provide a framework for measuring the expected value of reducing vagueness (EVRV) on a probability assessment using equivalent sample size and relative information multiple (RIM). We use influence diagrams to represent the flow of potential but unobserved evidence from a vagueness-reducing activity. We demonstrate our results for a simple problem implemented in Demos, a decision modeling software environment.