Many prior studies with attempt to improve regression testing adopt test case prioritization (TCP). TCP generally arranges the execution of regression test cases according to specific rules with the goal of revealing faults as early as possible. It is noted that different TCP algorithms adopt different metrics to evaluate test cases' priority so that they may be effect at revealing faults early in different faulty programs. Adopting a single metric may not generally work well. In this decade, learning-to-rank (LTR) strategies have been adopted to address some software engineering problems. This study also uses a pairwise LTR strategy XGBoost to combine several existing metrics so as to improve TCP effectiveness. More specifically, we regard the metrics adopted by TCP techniques to evaluate test cases' priority as the features of the training data and adopt XGBoost to learn the weights of the combined metrics. Additionally, in order to avoid overfitting, we use a fuzzy inference system to generate additional features for data augmentation. The experimental results show that our approach achieves more excellent effectiveness than the existing TCP techniques with respect to the selected subject programs.

Open source software (OSS) approach has become increasingly prevalent for software development. As the widespread utilization of OSS, the reliability of OSS products becomes an important issue. By simulating the testing and debugging processes of software life cycle, the rate-based queueing simulation model has shown its feasibility for closed source software (CSS) reliability assessment. However, the debugging activities of OSS projects are different in many ways from those of CSS projects and thus the simulation approach needs to be calibrated for OSS projects. In this paper, we first characterize the debugging activities of OSS projects. Based on this, we propose a new rate-based queueing simulation framework for OSS reliability assessment including the model and the procedures. Then a decision model is developed to determine the optimal version-updating time with respect to two objectives: minimizing the time for version update, and maximizing OSS reliability. To illustrate the proposed framework, three real datasets from Apache and GNOME projects are used. The empirical results indicate that our framework is able to effectively approximate the real scenarios. Moreover, the influences of the core contributor staffing levels are analyzed and the optimal version-updating times are obtained.

Test case prioritization techniques schedule the test cases in an order based on some specific criteria so that the tests with better fault detection capability are executed at an early position in the regression test suite. Many existing test case prioritization approaches are code-based, in which the testing of each software version is considered as an independent process. Actually, the test results of the preceding software versions may be useful for scheduling the test cases of the later software versions. Some researchers have proposed history-based approaches to address this issue, but they assumed that the immediately preceding test result provides the same reference value for prioritizing the test cases of the successive software version across the entire lifetime of the software development process. Thus, this paper describes ongoing research that studies whether the reference value of the immediately preceding test results is version-aware and proposes a test case prioritization approach based on our observations. The experimental results indicate that, in comparison to existing approaches, the presented one can schedule test cases more effectively.

This paper is an attempt to relax and improve the assumptions regarding software reliability modeling. To approximate reality much more closely, we take into account the concepts of testing compression factor and the quantified ratio of faults to failures in the modeling. Numerical examples based on real failure data show that the proposed framework has a fairly good prediction capability. Further, we also address the optimal software release time problem and conduct a detailed sensitivity analysis through the proposed model.

Research in the field of software reliability, dedicated to the analysis of software failure processes, is quite diverse. In recent years, several attractive rate-based simulation approaches have been proposed. Thus far, it appears that most existing simulation approaches do not take into account the number of available debuggers (or developers). In practice, the number of debuggers will be carefully controlled. If all debuggers are busy, they may not address newly detected faults for some time. Furthermore, practical experience shows that fault-removal time is not negligible, and the number of removed faults generally lags behind the total number of detected faults, because fault detection activities continue as faults are being removed. Given these facts, we apply the queueing theory to describe and explain possible debugging behavior during software development. Two simulation procedures are developed based on G/G/infin, and G/G/m queueing models, respectively. The proposed methods will be illustrated using real software failure data. The analysis conducted through the proposed framework can help project managers assess the appropriate staffing level for the debugging team from the standpoint of performance, and cost-effectiveness.

With the growing scale of software system, assuring software quality through automated testing becomes increasingly important. When automated testing is involved in software development, the uncertainty caused by automated test failures should not be ignored. Besides, the modification of tested software may introduce some potential faults and further invalidate some test scripts, which may lead to the failed outcomes. Based on the facts, we will propose a Markov renewal process (MRP) to model the correlation among software runs during the software development. The use of the proposed modeling framework is illustrated through an example. Compared to previous work, the proposed framework indeed addresses the influence of test automation and provides more useful information.

Summary form only given, as follows. The complete presentation was not made available for publication as part of the conference proceedings. Among many researches focusing on the prediction of software failure processes, rate-based simulation approaches are attractive in recent years. But only few existing simulation approaches consider the size of debugging team. In reality, the number of debuggers is always limited. If all debuggers are busy, the new detected faults should be willing to wait for a long time to be corrected. Practical experiences also show that the number of debugging personnel is tightly related to the service level of debugging system. Besides, the fault removal time should be non-negligible, and the phenomenon of imperfect debugging is inevitable in practice. To reflect these facts, in this paper, we propose a simulation-based approach to describe the possible debugging activities based on G/G/m queuing model. The imperfect and explicit debugging will also be taken into account in the proposed framework. In the experiments, a real data set is used to illustrate the proposed framework in detail. Experimental results will greatly help to analyze the influence of scale of debugging teams on the software failure correction activities and other related reliability assessments. Accordingly, project managers can have a guidance to strike the balance between the cost of debugging team and the progress of fault removals. 566

In the past decade, several rate-based simulation approaches were proposed to predict software failure process. But most of them did not take the number of available debuggers into consideration and this may not be reasonable. In practice, the number of debuggers is always limited and controlled. If all debuggers or developers are busy, the new detected faults should be willing to wait (for a long time to be corrected and removed). Besides, practical experiences also show that the fault removal time is non-negligible and the number of removed faults generally lags behind the total number of detected faults. Based on these facts, in this paper, we will apply queueing theory to describe and explain the possible debugging behavior during software development. Two simulation procedures are developed based on G/G/ infin and G/G/m queueing models. The proposed methods will be illustrated with real software failure data. Experimental results will be analyzed and discussed in detail. The results we obtained will greatly help to understand the influence of size of debugger teams on the software failure correction activities and other related reliability assessments.

Over the past 30 years, many software reliability growth models (SRGM) have been proposed. Often, it is assumed that detected faults are immediately corrected when mathematical models are developed. This assumption may not be realistic in practice because the time to remove a detected fault depends on the complexity of the fault, the skill and experience of personnel, the size of debugging team, the technique(s) being used, and so on. During software testing, practical experiences show that mutually independent faults can be directly detected and removed, but mutually dependent faults can be removed iff the leading faults have been removed. That is, dependent faults may not be immediately removed, and the fault removal process lags behind the fault detection process. In this paper, we will first give a review of fault detection & correction processes in software reliability modeling. We will then illustrate the fact that detected faults cannot be immediately corrected with several examples. We also discuss the software fault dependency in detail, and study how to incorporate both fault dependency and debugging time lag into software reliability modeling. The proposed models are fairly general models that cover a variety of known SRGM under different conditions. Numerical examples are presented, and the results show that the proposed framework to incorporate both fault dependency and debugging time lag for SRGM has a better prediction capability. In addition, an optimal software release policy for the proposed models, based on cost-reliability criterion, is proposed. The main purpose is to minimize the cost of software development when a desired reliability objective is given

In modern societies, software is everywhere and we need software to be reliable. In practice, during software development processes, software reliability assessment can greatly help managers to understand effectiveness of consumed testing-effort and deploy testing-resource. In the 1970s-2000, many software reliability growth models (SRGMs) have been proposed for estimation of reliability growth of software products. In this paper, the concept of multiple change-points is incorporated into Weibull-type testing-effort dependent SRGM because the consumption phenomenon of testing resource may vary at some moments. The performance and application of proposed models are demonstrated through one real data set. The experimental results show that the models give an excellence performance on failure prediction. Besides, we also discuss the optimal release time problems based on reliability requirement and cost criteria.

In this paper, we investigate some techniques for reliability prediction and assessment of fielded software. We first review how several existing software reliability growth models based on non-homogeneous Poisson processes (NHPPs) can be readily derived based on a unified theory for NHPP models. Furthermore, based on the unified theory, we can incorporate the concept of multiple change-points into software reliability modeling. Some models are proposed and discussed under both ideal and imperfect debugging conditions. A numerical example by using real software failure data is presented in detail and the result shows that the proposed models can provide fairly good capability to predict software operational reliability.

Since the early 1970s tremendous growth has been seen in the research of software reliability growth modeling. In general, software reliability growth models (SRGMs) are applicable to the late stages of testing in software development and they can provide useful information about how to improve the reliability of software products. For most existing SRGMs, most researchers assume that faults are immediately detected and corrected. However, in practice, this assumption may not be realistic and satisfied. In this paper we first give a review of fault detection and correction processes in SRGMs. We show how several existing SRGMs based on NHPP models can be comprehensively derived by applying the time-dependent delay function. Furthermore, we show how to incorporate both failure dependency and time-dependent delay function into software reliability growth modeling. We present stochastic reliability models for software failure phenomenon based on NHPPs. Some numerical examples based on real software failure data sets are presented. The results show that the proposed framework to incorporate both failure dependency and time-dependent delay function into software reliability modeling has a useful interpretation in testing and correcting the software.

Software reliability is defined as the probability of failure-free software operation for a specified period of time in a specified environment. Over the past 30 years, many software reliability growth models (SRGMs) have been proposed and most SRGMs assume that detected faults are immediately corrected. Actually, this assumption may not be realistic in practice. We first give a review of fault detection and correction processes in software reliability modeling. Furthermore, we show how several existing SRGMs based on NHPP models can be derived by applying the time-dependent delay function. On the other hand, it is generally observed that mutually independent software faults are on different program paths. Sometimes mutually dependent faults can be removed if and only if the leading faults were removed. Therefore, here we incorporate the ideas of fault dependency and time-dependent delay function into software reliability growth modeling. Some new SRGMs are proposed and several numerical examples are included to illustrate the results. Experimental results show that the proposed framework to incorporate both fault dependency and time-dependent delay function for SRGMs has a fairly accurate prediction capability.