ARE (absolute relative error) and SqRE (squared relative error), are random variables that are suggested as measurements of forecast accuracy of the total number of estimated software failures at the end of a mission time. The purpose is to compare the predictive merit of competing software reliability models, an important concern to software reliability analysts. This technique calculates the Bayes probability of how much better the prediction accuracy is for one method relative to a competitor. This novel approach is more realistic, in the assessment of predictive merit, than (a) comparing merely the average values of ARE and SqRE as conventionally done; and (b) conducting statistical hypothesis tests of pair-wise means of ARE and SqRE, an approach somewhat more sensible than (a), because (b) incorporates variability of predicted values, which (a) does not. To implement this technique, first noninformative (across the border) are used and then informative (specified) priors. For the informative case, half-normal priors are placed on the mean of the ARE or SqRE random variables, because these means are hypothesized to remain peaked around zero relative-error (ideal error percentage). This problem is related to the general problem of ranking usual means discussed in the literature by Berger and Deely (1988), and is a follow-up to an invited research paper presented at ISI-97 by Sahinoglu and Capar (1997).

If the recovery or remedial time is not incorporated in the reliability of a software module in a safety and time-critical integrated system operation, then a mere reliability index based on failure characteristics is simply not adequate and realistic. In deriving the probability density function (pdf) of the software availability, empirical Bayesian procedures will be used to employ expert engineering judgment through appropriate noninformative and informative prior distribution functions employing various definitions of risk functions. It is emphasized that the usage of maximum likelihood estimators regardless of the extent of historical data is erroneous and misleading.

The ‘compound Poisson’ (CP) software reliability model was proposed previously by the first named author for time-between-failure data in terms of CPU seconds, using the ‘maximum likelihood estimation’ (MLE) method to estimate unknown parameters; hence, CPMLE. However, another parameter estimation technique is proposed under ‘nonlinear regression analysis’ (NLR) for the compound Poisson reliability model, giving rise to the name CPNLR. It is observed that the CP model, with different parameter estimation methods, produces equally satisfactory or more favourable results as compared to the Musa–Okumoto (M–O) model, particularly in the event of grouped or clustered (clumped) software failure data. The sampling unit may be a week, day or month within which the failures are clumped, as the error recording facilities dictate in a software testing environment. The proposed CPNLR and CPMLE yield comparatively more favourable results for certain software failure data structures where the frequency distribution of the cluster (clump) size of the software failures per week displays a negative exponential behaviour. Average relative error (ARE), mean squared error (MSE) and average Kolmogorov–Smirnov (K–S Av.Dn) statistics are used as measures of forecast quality for the proposed and competing parameter-estimation techniques in predicting the number of remaining future failures expected to occur until a target stopping time. Comparisons on five different simulated data sets that contain weekly recorded software failures are made to emphasize the advantages and disadvantages of the competing methods by means of the chronological prediction plots around the true target value and zero per cent relative error line. The proposed generalized compound Poisson (MLE and NLR) methods consistently produce more favourable predictions for those software failure data with negative exponential frequency distribution of the failure clump size versus number of weeks. Otherwise, the popularly used competing M–O log-Poisson model is a better fit for those data with a uniform clump size distribution to recognize the log-Poisson effect while the logarithm of the Poisson equation is a constant, hence uniform. The software analyst is urged to perform exploratory data analysis to recognize the nature of the software failure data before favouring a particular reliability estimation method. © 1997 by John Wiley & Sons, Ltd.

The probability density estimation of the number of software failures in the event of clustering or clumping of the software failures is considered. A discrete compound Poisson (CP) prediction model is proposed for the random variable X/sub rem/, which is the remaining number of software failures. The compounding distributions, which are assumed to govern the failure sizes at Poisson arrivals, are respectively taken to be geometric when failures are forgetful and logarithmic-series when failures are contagious. The expected value ( mu ) of X/sub rem/ is calculated as a function of the time-dependent Poisson and compounding distribution based on the failures experienced. Also, the variance/mean parameter for the remaining number of failures, q/sub rem/, is best estimated by q/sub past/ from the failures already experienced. Then, one obtains the PDF of the remaining number of failures estimated by CP( mu ,q). CP is found to be superior to Poisson where clumping of failures exists. Its predictive validity is comparable to the Musa-Okumoto log-Poisson model in certain cases.>