In software debugging, fault localization (FL) is an essential stage that is used to identify accurate location of faulty statement. It is well known that coverage data plays an important role in FL. In past studies, traditional principal component analysis (PCA) or revised PCA techniques were used to reduce coverage data. However, two kinds of PCA have a great opportunity to remove the actual faulty statement, especially in multiple fault localization. On the other hand, they cannot reflect the status of the statements. In this paper, we propose a novel approach based on revised PCA and incorporate the result of failed and passed test case different combinations to update contribution value of each statement. We used two Linux open-source codes (Sed, Grep) with 4 fault injections to verify the correctness. Preliminary experiments have shown that our proposed method is feasible, scalable, and shorter execution time of FL process, and also can alleviate the situations for removed faulty statements compared to the revised PCA method.

Software defect prediction (SDP) is an essential technique for identifying potential defects in software projects. Generally, SDP is mainly divided into two procedures: extracting features from source code and building a classification model using machine learning methods. However, SDP contends with specific limitations. For example, machine learning models require a fixed input size, but the size of each program is mostly inconsistent. Another limitation is that the amount of training data may be too small for machine learning, and it is extremely difficult to handle class imbalance and dataset expansion. In this study, we propose a method called spatial pyramid pooling for defect prediction (SPP-DP) that first converts all the source files into images, each of which will generate multiple images with different lengths and widths, to address the limitations of class imbalance handling and data augmentation. Second, we input these images into a convolutional neural network (CNN) to build a classifier to predict software defects. We added a spatial pyramid pooling layer (SPP-Layer) architecture to the CNN to relax the limitation of the fixed input size. Compared with different deep learning-based techniques on five datasets, the experimental results show that our proposed SPP-DP is effective, as it can balance the dataset and provide better software defect identification ability.

Software defect prediction (SDP) method aims to identify potential bugs in programs or defective modules in software projects. SDP method can greatly help developers allocate needed testing- and debugging-efforts. Presently, SDP is typically divided into two procedures: extracting features from source code and building a classification model using machine learning methods, such as support vector machine, decision tree, and neural networks, to build a classifier for defect prediction. However, there are still some limitations for SDP. For example, machine learning models require a fixed input size, but the size of each program is mostly inconsistent. Activation functions play an essential role in the training of artificial neural networks, but every kind of activation function has its own particular strengths and inherent constraints. The main purpose of this article is to propose a general framework of combining different activation functions with given weights to improve the effectiveness of SDP. We construct 41 kinds of defect prediction models by deep belief networks (DBNs) built with double-weighted or triple-weighted combination of six most commonly used activation functions to improve the predictions. In the experiment of this study, some real data from open-source projects are selected to evaluate the performance of our proposed weighted combination methods. It is found experimentally that the weighted combinations methods can enhance the accuracy of SDP. It is also noticed that our proposed weighted combination scheme is not restricted to the DBN or a particular kind of activation functions.

In contemporary software development processes, as the software development cycle undergoes continuous evolution, the subsequent maintenance phase often veers away from the original architectural plan. While clustering serves as an intuitive approach for implementing modularization, traditional clustering algorithms exhibit certain limitations, such as generating excessively large output modules. This paper introduces a novel multi-pattern modularity clustering algorithm (MPMC) aimed at enhancing the quality of software modularization. This algorithm hinges on two key components. Firstly, it employs a multi-pattern strategy to scrutinize the extent of dependencies between files. This involves distinct steps like tagging files, aggregating chain dependencies, and subsequently preprocessing and amalgamating the modules. Secondly, it incorporates a multi-level modularity clustering algorithm, which employs a graph partitioning technique to categorize graphs into clusters through processes like coarsening and multi-level refinement. To evaluate the performance of MPMC, five open-source and one closed-source programs, each varying in size and functionality, were employed for comparison against other conventional software grouping algorithms and existing software clustering methods. Experimental results reveal a substantial improvement in the performance of MPMC, approximately 2.71 times higher in terms of weighted modularity quality (MQ) criteria compared to alternative methods, and a 1.35 times improvement in the MoJoFM criteria. These findings establish the effectiveness of the MPMC algorithm as a valuable software clustering tool for developers, emphasizing its ability to yield superior module quality in comparison to other methods.

Abstract In software engineering, a software development process, also known as software development life cycle (SDLC), involves several distinct activities for developing, testing, maintaining, and evolving a software system. Within the stages of SDLC, software maintenance occupies most of the total cost of the software life. However, after extended maintenance activities, software quality always degrades due to increasing size and complexity. To solve this problem, software modularisation using clustering is an intuitive way to modularise and classify code into small pieces. , A multi‐pattern clustering (MPC) algorithm for software modularisation is proposed in this study. The proposed MPC algorithm can be divided into five different steps: (1) preprocessing, (2) file labelling, (3) collection of chain dependencies, (4) hierarchical agglomerative clustering, (5) modification of the clustering result. The performance of the proposed MPC algorithm to selected clustering techniques is compared by using three open‐source and one closed‐source software programs. Experimental results show that the modularisation quality of the proposed MPC algorithm is nearly 1.6 times better than that of the expert decomposition. Additionally, compared to other software clustering algorithms, the proposed MPC algorithm, on average, has a 13% enhancement in producing results similar to human thinking. Consequently, it can be seen that the proposed MPC algorithm is suitable for human comprehension while producing better module quality compared to other clustering algorithms.

Over the past few decades, researchers in software engineering (SE) have focused on testing, analyzing, repairing, and generating programs automatically and effectively. Today, combining neural networks and traditional software engineering techniques has major potential to benefit software quality and productivity. Regarding the development of neural networks, deep learning (DL) and convolution neural networks (CNNs) have been widely adopted by software applications for making decisions or providing suggestions. Considering life-critical DL-based applications, there is a need to correct the wrong decisions made by DL systems immediately. Therefore, we propose a novel fault-correction framework for alleviating potential misclassification issues of DL systems called the Outlier Modification for DL Systems (OMDLS). Our experiment results with two public datasets using different scales and label numbers to show that modifying the outliers based on the misclassification pairs can improve accuracy by up to 2.12% without retraining the model and modifying the inference immediately.

In the popularity of the Internet, users can find a variety of services on the Internet to meet their needs; but whether the stability of software service is a problem for users. Similarly, service providers seek to continuously update service to provide a better user experience. In this paper, we proposed four QoS prediction architectures that can consider more factors and variables than past methods. We compared the prediction performance of our proposed models with the regression model, the convolutional neural network (CNN), and the recurrent neural network (RNN). Four real datasets are used and experimental results show that considering multiple variables are better than single variable as inputs in our models. The Single method performed better than past methods using a single time series. Moreover, in four datasets multi-factor method predicted better than single factor method. Each model took different average training time in different datasets. Some methods took less time but does not have good performance. We used a CNN and RNN to converge more quickly than long short-term memory (LSTM) and gated recurrent unit (GRU), and to also achieve good prediction performance.

Service-oriented architecture (SOA) has become an increasingly popular choice for building software application in the last years. An SOA system is an elastic structure that utilizes services discovery and integrates these services to perform specified functions. In general, reliability is a critical system attribute when evaluating the quality of a well-built software applications. But it has to be noted that the phenomenon of error propagation could have significant impacts on system reliability. Propagated errors may be masked or propagated to the system interface, which can thereby lead to a system failure. Much research on reliability evaluation for SOA systems have been proposed in the past. However, most of these studies have neglected the phenomenon of error propagation and the issue of link failure. In this article, we take a different view of error propagation, fault tolerance, and the failure behavior of links between services and try to develop an enhanced SOA reliability prediction model incorporating error propagation and fault tolerance (EP-FT). Different fault tolerance techniques will be selected and integrated into the SOA systems. Additionally, sensitivity analysis is also presented and discussed to determine the critical services in the system. Experiments are performed based on four real-world case studies. We will show and discuss the experimental results of the proposed EP-FT reliability model and simulation-based approach in detail. Our experimental results show that the impact of error propagation on system reliability is not negligible, and the SOA systems with fault tolerance demonstrate higher reliability than those that do not.

Software is widely used in many application domains. The most popular software are used by millions every day. How to accurately predict and assess the reliability of developed software is becoming increasingly important for project managers and developers. Previous studies have primarily used the software reliability growth model (SRGM) to evaluate and predict software reliability, but prediction results cannot be accurate at particular times or in particular situations. One of the main reasons is that simplified assumptions and abstractions are usually made to simplify the problem when developing SRGMs. Selecting an appropriate SRGM should depend on the key characteristics of the software project. In this article, we propose a deep learning-based approach for software reliability prediction and assessment. Specifically, we clearly demonstrate how to derive mathematical expressions from the computational methods of deep learning models and how to determine the correlation between them and the mathematical formula of SRGMs, and then, we use the back-propagation algorithm to obtain the SRGM parameters. Furthermore, we further integrate some deep learning-based SRGMs and also propose a method for the weighted assignment of combinations. Three real open source software failure datasets are used to evaluate the performance of the proposed models compared to selected SRGMs. The experimental results reveal that our proposed deep learning-based models and their combinations perform better than several classical SRGMs.

Service-based software systems (SBSSs) are widely deployed due to the growing trend of distributed computing and cloud computing. It is important to ensure high quality of an SBSS, especially in a strongly competitive market. Existing works on SBSS reliability usually assumed independence of service failures. However, the fact that resource sharing exists in different levels of SBSS operations invalidates this assumption. Ignorance of failure dependencies have been discussed as potentially affecting system reliability predictions and lowering the benefits of design diversity, as typically seen in high-reliability systems. In this paper, we propose a reliability framework that incorporates failure dependence modeling, system reliability modeling, as well as reliability analysis for individual services and for failure sources. The framework is also capable of analyzing the internal structures of popular software fault tolerant (FT) schemes. The proposed method is applied to a travel agency system based upon a real-world practice for verifying its accuracy of reliability modeling and effectiveness of varied reliability measures. The results show that failure dependence of the services is an essential factor for analyzing any valuable SBSS system. Further, a set of reliability measures with different capabilities and complexities are available for assisting SBSS engineers with system improvements.

Summary Presently, service‐based software systems (SBSSs) have been heavily deployed to fulfil the functionalities of cloud computing and are widely used in many other application fields. Additionally, maintaining functionality and quality of service levels becomes increasingly important for SBSSs; this is because system operational failures may cause great financial loss to an organization. Fault tolerance (FT) is usually used to provide continuous and reliable system service delivery when failures occur. However, the reliability and performance of FT should be carefully analysed because of the overhead of invoking redundant services. It is also noted that the single point of failure on the FT adjudicators as well as the failure correlation also hamper the benefits of FT in SBSSs. To address these problems, this paper proposes two approaches, the stochastic modelling approach and the simulation approach, for analysing the reliability and performance of generalized FT designs. The first approach is suitable for quick analysis at an early design stage, while the second approach is built on top of the ns‐3 simulator and could be well adapted to incorporate varied uncertainty models in the SBSS environments Extensive experiments and analyses uncover some characteristics that could be useful for SBSS engineers. Copyright © 2015 John Wiley & Sons, Ltd.

Software systems are widely employed in society. With a limited amount of testing resource available, testing resource allocation among components of a software system becomes an important issue. Most existing research on the testing resource allocation problem takes a single-objective optimization approach, which may not adequately address all the concerns in the decision-making process. In this paper, an architecture-based multi-objective optimization approach to testing resource allocation is proposed. An architecture-based model is used for system reliability assessment, which has the advantage of explicitly considering system architecture over the reliability block diagram (RBD)-based models, and has good flexibility to different architectural alternatives and component changes. A system cost modeling approach which is based on well-developed software cost models is proposed, which would be a more flexible, suitable approach to the cost modeling of software than the approach adopted by others which is based on an empirical cost model. A multi-objective optimization model is developed for the testing resource allocation problem, in which the three major concerns in the testing resource allocation problem, i.e., system reliability, system cost, and the total amount of testing resource consumed, are taken into consideration. A multi-objective evolutionary algorithm (MOEA), called multi-objective differential evolution based on weighted normalized sum (WNS-MODE), is developed. Experimental studies are presented, and the experiments show several results. 1) The proposed architecture-based multi-objective optimization approach can identify the testing resource allocation strategy which has a good trade-off among optimization objectives. 2) The developed WNS-MODE is better than the MOEA developed in recent research, called HaD-MOEA, in terms of both solution quality and computational efficiency. 3) The WNS-MODE seems quite robust from the sensitivity analysis results.

SUMMARY In the modern society, software plays a very important role in many application systems. Consequently, the main goal of project managers and software engineers is to deliver reliable software within very limited resource, time and budget during the software development life cycle. Presently, it is widely recognized that open source software (OSS) has developed as a new (and novel) form of both personal and aggregation production. In the past, some research has shown that the traditional Pareto distribution (PD) and the Weibull distribution models can be used to describe the distribution of software faults of OSS. However, there could be a negative value for the cumulative probability of the traditional PD model in some cases. In this paper, based on our past studies, a modified Pareto‐based distribution model, called the single‐change‐point 2‐parameter generalized PD (SCP‐2GPD) model is proposed. The method of choosing an appropriate change‐point is presented and illustrated. Some mathematical properties of the proposed model are also discussed. Experiments are conducted using several real OSS data, and evaluation results show that our proposed SCP‐2GPD model depicts the real‐life situation of the software development life cycle more faithfully and accurately. Copyright © 2013 John Wiley & Sons, Ltd.

Software is currently a key part of many safety-critical and life-critical application systems. People always need easy- and instinctive-to-use software, but the biggest challenge for software engineers is how to develop software with high reliability in a timely manner. To assure quality, and to assess the reliability of software products, many software reliability growth models (SRGMs) have been proposed in the past three decades. The practical problem is that sometimes these selected SRGMs by companies or software practitioners disagree in their reliability predictions, while no single model can be trusted to provide consistently accurate results across various applications. Consequently, some researchers have proposed to use combinational models for improving the prediction capability of software reliability. In this paper, three enhanced weighted-combinations, namely weighted arithmetic, weighted geometric, and weighted harmonic combinations, are proposed. To solve the problem of determining proper weights for model combinations, we further study how to incorporate enhanced genetic algorithms (EGAs) with several efficient operators into weighted assignments. Experiments are performed based on real software failure data, and numerical results show that our proposed models are flexible enough to depict various software development environments. Finally, some management metrics are presented to both assure software quality and determine the optimal release strategy of software products under development.

In general, one of the most important aspects of software development and project management is how to make predictions and assessments of quality and reliability for developed products. Project data usually will be systematically collected and analyzed during the process of software development. Practically, it would be helpful if developers could identify the most error-prone modules early so that they can optimize testing-resource allocation and increase fault detection effectiveness accordingly. In the past, many research studies revealed the applicability of the Pareto principle to software systems, and some of them reported that the Pareto distribution (PD) model can be used to predict the fault distribution of software. In this paper, a special form of the Generalized PD model, named the Bounded Generalized Pareto distribution (BGPD) model, is further proposed to investigate the fault distributions of Open Source Software (OSS). It can be seen that the BGPD model eliminates the issue which occurred in the classical PD model. Three methods of parameter estimation will be presented, and related experiments are performed based on real OSS failure data. Experimental results show that the BGPD model presents high fitness to the actual failure data of OSS. Finally, the possibility of using early limited fault data to predict the later software fault distribution is also studied. Numerical results indicate that the BGPD model can be trusted to consistently produce accurate estimates of fault predictions during the early stages of development. The findings can provide an effective foundation for managing the necessary activities of software development and testing.

During the process of fault localization, the spectrum-based techniques are frequently used and widely studied since they can automatically and effectively localize the faults of software and be implemented easily. So far most of spectrum-based fault localization techniques have relied heavily on the use of similarity coefficients. However, we noticed that existing similarity coefficients for fault localization may lack measure(s) to properly reflect the relationship between failing and passing test cases. It also has to note that the failing test cases usually are expected to provide more information to the similarity coefficients than the passing test cases. In order to evaluate the importance of failing and passing test cases in the similarity coefficients, a number of modified similarity coefficients in fault localization are presented and discussed. The modified similarity coefficients which are assigned the weights of the failing and/or passing test cases will be studied and analyzed with the multi-coverage-combined techniques. Five open source programs and 75 faulty versions in total from Siemens suite, which have been widely used for software testing and comparison of fault localization techniques, were selected as experiment subjects. Detailed analysis of the results shows that assigning the weights of failing and passing test cases to the similarity coefficients would be able to localize the faults more effectively and accurately.

With the growing size and complexity of software applications, traditional software reliability methods are insufficient to analyze inter-component interactions of modular software systems. The number of test cases may be extremely large for this application; therefore, it is hard for us to extensively test each software component given resource limitations. In this paper, we propose an adaptive framework of incorporating path testing into reliability estimation for modular software systems. Three estimated methods based on common program structures, namely, sequence, branch, and loop structures, are proposed to calculate the path reliability. Consequently, the derived path reliabilities can be applied to the estimates of software reliability. Some experiments are performed based on two real systems. In addition, the accuracy and correlation with respect to the experiments are investigated by simulation and sensitivity analysis. Experimental results show that the path reliability has a high correlation to the actual software reliability. For software with loop structures, a smaller loop number can be assigned to derive an acceptable estimation of path reliability. Further, the sensitivity analysis can be used to identify critical modules and paths for resource allocation. It can be concluded that the proposed methods are useful and helpful for estimating software reliability and can be adaptively used in the early stages of software development.

Software typically undergoes debugging during both a testing phase before product release, and an operational phase after product release. But it is noted that the fault detection and removal processes during software development and operation are different. For example, the fault removal during operation occurs generally at a slower pace than development. In this paper, we derive a powerful, easily deployable technique for software reliability prediction and assessment in the testing and operational phases. We first review how several existing software reliability growth models (SRGM) based on non- homogeneous Poisson processes (NHPP) can be readily derived from a unified theory. With the unified theory, we further incorporate the concept of multiple change-points, i.e. points in time when the software environment changes, into software reliability modeling. Several models are proposed and discussed under both ideal and imperfect debugging conditions. We estimate the parameters of the proposed models by employing real software failure data, and give a fair comparison with some existing SRGM. Numerical results show that the proposed models can provide good software reliability prediction in the various stages of software development and operation. Our approach is flexible; we can model various environments ranging from exponential-type to S-shaped NHPP models.

In order to assure software quality and assess software reliability, many software reliability growth models (SRGMs) have been proposed for estimation of reliability growth of products in the past three decades. In principle, two widely used methods for the parameter estimation of SRGMs are the maximum likelihood estimation (MLE) and the least squares estimation (LSE). However, the approach of these two estimations may impose some restrictions on SRGMs, such as the existence of derivatives from formulated models or the needs for complex calculation. Thus in this paper, we propose a modified genetic algorithm (MGA) to estimate the parameters of SRGMs. Experiments based on real software failure data are performed, and the results show that the proposed genetic algorithm is more effective and faster than traditional genetic algorithms.

During software development, regression testing is usually used to assure the quality of modified software. The techniques of test case prioritization schedule the test cases for regression testing in an order that attempts to increase the effectiveness in accordance with some performance goal. The most general goal is the rate of fault detection. It assumes all test case costs and fault severities are uniform. However, those factors usually vary. In order to produce a more satisfactory order, the cost-cognizant metric that incorporates varying test case costs and fault severities is proposed. In this paper, we propose a cost-cognizant test case prioritization technique based on the use of historical records and a genetic algorithm. We run a controlled experiment to evaluate the proposed technique's effectiveness. Experimental results indicate that our proposed technique frequently yields a higher Average Percentage of Faults Detected per Cost (APFDc). The results also show that our proposed technique is also useful in terms of APFDc when all test case costs and fault severities are uniform.

This paper is an attempt to relax and improve the assumptions regarding software reliability modeling. To approximate reality much more closely, we take into account the concepts of testing compression factor and the quantified ratio of faults to failures in the modeling. Numerical examples based on real failure data show that the proposed framework has a fairly good prediction capability. Further, we also address the optimal software release time problem and conduct a detailed sensitivity analysis through the proposed model.

In the past, some researches suggested that engineers can use combined software reliability growth models (SRGMs) to obtain more accurate reliability prediction during testing. In this paper, three weighted combinational models, namely, equal, linear, and nonlinear weight, are proposed for reliability estimation of web-based software. We further investigate the estimation accuracy of using genetic algorithm to determine the weight assignment for the proposed models. Preliminary result shows that the linearly and nonlinearly weighted combinational models have better prediction capability than single SRGM and equally weighted combinational model for web-based software.

Research in the field of software reliability, dedicated to the analysis of software failure processes, is quite diverse. In recent years, several attractive rate-based simulation approaches have been proposed. Thus far, it appears that most existing simulation approaches do not take into account the number of available debuggers (or developers). In practice, the number of debuggers will be carefully controlled. If all debuggers are busy, they may not address newly detected faults for some time. Furthermore, practical experience shows that fault-removal time is not negligible, and the number of removed faults generally lags behind the total number of detected faults, because fault detection activities continue as faults are being removed. Given these facts, we apply the queueing theory to describe and explain possible debugging behavior during software development. Two simulation procedures are developed based on G/G/infin, and G/G/m queueing models, respectively. The proposed methods will be illustrated using real software failure data. The analysis conducted through the proposed framework can help project managers assess the appropriate staffing level for the debugging team from the standpoint of performance, and cost-effectiveness.

In this paper, we propose a modified genetic algorithm (MGA) with calibrating fitness functions, weighted bit mutation, and rebuilding mechanism for the parameter estimation of software reliability growth models (SRGMs). An example using a real failure data is given to demonstrate the performance of proposed method. Experimental result shows that MGA is effective for estimating the parameters of SRGM.

With the growing scale of software system, assuring software quality through automated testing becomes increasingly important. When automated testing is involved in software development, the uncertainty caused by automated test failures should not be ignored. Besides, the modification of tested software may introduce some potential faults and further invalidate some test scripts, which may lead to the failed outcomes. Based on the facts, we will propose a Markov renewal process (MRP) to model the correlation among software runs during the software development. The use of the proposed modeling framework is illustrated through an example. Compared to previous work, the proposed framework indeed addresses the influence of test automation and provides more useful information.

In software development and maintenance, locating faults is generally a complex and time-consuming process. In order to effectively identify the locations of program faults, several approaches have been proposed. Similarity-aware fault localization (SAFL) is a testing-based fault localization method that utilizes testing information to calculate the suspicion probability of each statement. Dicing is also another method that we have used. In this paper, our proposed method focuses on predicates and their influence, instead of on statements in traditional SAFL. In our method, fuzzy theory, matrix calculating, and some probability are used. Our method detects the importance of each predicate and then provides more test data for programmers to analyze the fault locations. Furthermore, programmers will also gain some important information about the program in order to maintain their program accordingly. In order to speed up the efficiency, we also simplified the program. We performed an experimental study for several programs, together with another two testing-based fault localization (TBFL) approaches. These three methods were discussed in terms of different criteria such as line of code and suspicious code coverage. The experimental results show that the proposed method from our study can decrease the number of codes which have more probability of suspicion than real bugs.

Software reliability is often defined as the probability of failure-free software operation for a specified period of time in a specified environment. During the past 30 years, many software reliability growth models (SRGM) have been proposed for estimating the reliability growth of software. In practice, effective debugging is not easy because the fault may not be immediately obvious. Software engineers need time to read, and analyze the collected failure data. The time delayed by the fault detection & correction processes should not be negligible. Experience shows that the software debugging process can be described, and modeled using queueing system. In this paper, we will use both finite, and infinite server queueing models to predict software reliability. We will also investigate the problem of imperfect debugging, where fixing one bug creates another. Numerical examples based on two sets of real failure data are presented, and discussed in detail. Experimental results show that the proposed framework incorporating both fault detection, and correction processes for SRGM has a fairly accurate prediction capability.

Grey relational analysis (GRA), a similarity-based method, presents acceptable prediction performance in software effort estimation. However, we found that conventional GRA methods only consider non-weighted conditions while predicting effort. Essentially, each feature of a project may have a different degree of relevance in the process of comparing similarity. In this paper, we propose six weighted methods, namely, non-weight, distance-based weight, correlative weight, linear weight, nonlinear weight, and maximal weight, to be integrated into GRA. Three public datasets are used to evaluate the accuracy of the weighted GRA methods. Experimental results show that the weighted GRA performs better precision than the non-weighted GRA. Specifically, the linearly weighted GRA greatly improves accuracy compared with the other weighted methods. To sum up, the weighted GRA not only can improve the accuracy of prediction but is an alternative method to be applied to software development life cycle.

Summary form only given, as follows. The complete presentation was not made available for publication as part of the conference proceedings. Among many researches focusing on the prediction of software failure processes, rate-based simulation approaches are attractive in recent years. But only few existing simulation approaches consider the size of debugging team. In reality, the number of debuggers is always limited. If all debuggers are busy, the new detected faults should be willing to wait for a long time to be corrected. Practical experiences also show that the number of debugging personnel is tightly related to the service level of debugging system. Besides, the fault removal time should be non-negligible, and the phenomenon of imperfect debugging is inevitable in practice. To reflect these facts, in this paper, we propose a simulation-based approach to describe the possible debugging activities based on G/G/m queuing model. The imperfect and explicit debugging will also be taken into account in the proposed framework. In the experiments, a real data set is used to illustrate the proposed framework in detail. Experimental results will greatly help to analyze the influence of scale of debugging teams on the software failure correction activities and other related reliability assessments. Accordingly, project managers can have a guidance to strike the balance between the cost of debugging team and the progress of fault removals. 566

The coverage criteria of verification techniques play an important role in software development and testing. The goal is to reduce the size of test suites to economize on time, and to ensure whether all statements (or conditions) are covered. We can use these criteria to track test progress, assess current situations, predict emerging events, and so on. Thus we can take the necessary actions upon early indications that testing activity is falling behind. Modified condition/decision coverage (MC/DC) was proposed by NASA in 1994, and had been widely adopted and discussed since then. As evident from the definition, MC/DC criterion is used to judge whether each Boolean operator can be satisfied or not. However, we find that the selected test cases sometimes may not be able to satisfy the original definition of MC/DC under some Boolean expressions. In this paper, we will propose a simple but useful method which focuses on all conditions of Boolean expression to practice MC/DC. Specifically, our proposed approach will use n-cube graph and Gray code to implement the MC/DC criterion. We will further show how to use the proposed method to differentiate the necessary and redundancy test cases. Finally, a practical regression testing tool, TASTE (Tool for Automatic Software regression TEsting), will be presented in this paper. An example is given to illustrate the detailed working process and is explained in detail.

In the past decade, several rate-based simulation approaches were proposed to predict software failure process. But most of them did not take the number of available debuggers into consideration and this may not be reasonable. In practice, the number of debuggers is always limited and controlled. If all debuggers or developers are busy, the new detected faults should be willing to wait (for a long time to be corrected and removed). Besides, practical experiences also show that the fault removal time is non-negligible and the number of removed faults generally lags behind the total number of detected faults. Based on these facts, in this paper, we will apply queueing theory to describe and explain the possible debugging behavior during software development. Two simulation procedures are developed based on G/G/ infin and G/G/m queueing models. The proposed methods will be illustrated with real software failure data. Experimental results will be analyzed and discussed in detail. The results we obtained will greatly help to understand the influence of size of debugger teams on the software failure correction activities and other related reliability assessments.

Over the last several decades, many Software Reliability Growth Models (SRGM) have been developed to greatly facilitate engineers and managers in tracking and measuring the growth of reliability as software is being improved. However, some research work indicates that the delayed S-shaped model may not fit the software failure data well when the testing-effort spent on fault detection is not a constant. Thus, in this paper, we first review the logistic testing-effort function that can be used to describe the amount of testing-effort spent on software testing. We describe how to incorporate the logistic testing-effort function into both exponential-type, and S-shaped software reliability models. The proposed models are also discussed under both ideal, and imperfect debugging conditions. Results from applying the proposed models to two real data sets are discussed, and compared with other traditional SRGM to show that the proposed models can give better predictions, and that the logistic testing-effort function is suitable for incorporating directly into both exponential-type, and S-shaped software reliability models.

Over the past 30 years, many software reliability growth models (SRGMs) have been proposed for estimation of reliability growth of software. In fact, effective debugging is not easy because the fault may not be immediately obvious. In the past, some researchers ever used an infinite server queueing (ISO) model to describe the software debugging behavior. An infinite-server queueing model is considered where access of customers to service is controlled by a gate and the gate is open only if all servers are free. However, the finite server queueing (FSQ) model is first advantageously modeled as an infinite-server system. Thus, in this paper, we show how to incorporate both FSQ and ISQ models into software reliability estimation and prediction. In addition, we also consider the factor of perfect/imperfect debugging. Experimental results show that the proposed framework to incorporate both fault detection and correction processes for SRGM has a fairly accurate prediction capability

Over the past 30 years, many software reliability growth models (SRGM) have been proposed. Often, it is assumed that detected faults are immediately corrected when mathematical models are developed. This assumption may not be realistic in practice because the time to remove a detected fault depends on the complexity of the fault, the skill and experience of personnel, the size of debugging team, the technique(s) being used, and so on. During software testing, practical experiences show that mutually independent faults can be directly detected and removed, but mutually dependent faults can be removed iff the leading faults have been removed. That is, dependent faults may not be immediately removed, and the fault removal process lags behind the fault detection process. In this paper, we will first give a review of fault detection & correction processes in software reliability modeling. We will then illustrate the fact that detected faults cannot be immediately corrected with several examples. We also discuss the software fault dependency in detail, and study how to incorporate both fault dependency and debugging time lag into software reliability modeling. The proposed models are fairly general models that cover a variety of known SRGM under different conditions. Numerical examples are presented, and the results show that the proposed framework to incorporate both fault dependency and debugging time lag for SRGM has a better prediction capability. In addition, an optimal software release policy for the proposed models, based on cost-reliability criterion, is proposed. The main purpose is to minimize the cost of software development when a desired reliability objective is given

In modern societies, software is everywhere and we need software to be reliable. In practice, during software development processes, software reliability assessment can greatly help managers to understand effectiveness of consumed testing-effort and deploy testing-resource. In the 1970s-2000, many software reliability growth models (SRGMs) have been proposed for estimation of reliability growth of software products. In this paper, the concept of multiple change-points is incorporated into Weibull-type testing-effort dependent SRGM because the consumption phenomenon of testing resource may vary at some moments. The performance and application of proposed models are demonstrated through one real data set. The experimental results show that the models give an excellence performance on failure prediction. Besides, we also discuss the optimal release time problems based on reliability requirement and cost criteria.

In this paper, we investigate some techniques for reliability prediction and assessment of fielded software. We first review how several existing software reliability growth models based on non-homogeneous Poisson processes (NHPPs) can be readily derived based on a unified theory for NHPP models. Furthermore, based on the unified theory, we can incorporate the concept of multiple change-points into software reliability modeling. Some models are proposed and discussed under both ideal and imperfect debugging conditions. A numerical example by using real software failure data is presented in detail and the result shows that the proposed models can provide fairly good capability to predict software operational reliability.

In this paper, we study the impact of software testing effort & efficiency on the modeling of software reliability, including the cost for optimal release time. This paper presents two important issues in software reliability modeling & software reliability economics: testing effort, and efficiency. First, we propose a generalized logistic testing-effort function that enjoys the advantage of relating work profile more directly to the natural flow of software development, and can be used to describe the possible testing-effort patterns. Furthermore, we incorporate the generalized logistic testing-effort function into software reliability modeling, and evaluate its fault-prediction capability through several numerical experiments based on real data. Secondly, we address the effects of new testing techniques or tools for increasing the efficiency of software testing. Based on the proposed software reliability model, we present a software cost model to reflect the effectiveness of introducing new technologies. Numerical examples & related data analyzes are presented in detail. From the experimental results, we obtain a software economic policy which provides a comprehensive analysis of software based on cost & test efficiency. Moreover, the policy can also help project managers determine when to stop testing for market release at the right time.

We consider two kinds of software testing-resource allocation problems. The first problem is to minimize the number of remaining faults given a fixed amount of testing-effort, and a reliability objective. The second problem is to minimize the amount of testing-effort given the number of remaining faults, and a reliability objective. We have proposed several strategies for module testing to help software project managers solve these problems, and make the best decisions. We provide several systematic solutions based on a nonhomogeneous Poisson process model, allowing systematic allocation of a specified amount of testing-resource expenditures for each software module under some constraints. We describe several numerical examples on the optimal testing-resource allocation problems to show applications & impacts of the proposed strategies during module testing. Experimental results indicate the advantages of the approaches we proposed in guiding software engineers & project managers toward best testing resource allocation in practice. Finally, an extensive sensitivity analysis is presented to investigate the effects of various principal parameters on the optimization problem of testing-resource allocation. The results can help us know which parameters have the most significant influence, and the changes of optimal testing-effort expenditures affected by the variations of fault detection rate & expected initial faults.

Since the early 1970s tremendous growth has been seen in the research of software reliability growth modeling. In general, software reliability growth models (SRGMs) are applicable to the late stages of testing in software development and they can provide useful information about how to improve the reliability of software products. For most existing SRGMs, most researchers assume that faults are immediately detected and corrected. However, in practice, this assumption may not be realistic and satisfied. In this paper we first give a review of fault detection and correction processes in SRGMs. We show how several existing SRGMs based on NHPP models can be comprehensively derived by applying the time-dependent delay function. Furthermore, we show how to incorporate both failure dependency and time-dependent delay function into software reliability growth modeling. We present stochastic reliability models for software failure phenomenon based on NHPPs. Some numerical examples based on real software failure data sets are presented. The results show that the proposed framework to incorporate both failure dependency and time-dependent delay function into software reliability modeling has a useful interpretation in testing and correcting the software.

This paper addresses the problem of achieving full restoration and tolerating as many faults as possible in wavelength division multiplexing (WDM) networks without the capability of wavelength conversion. The problem of finding the maximum number of faults that can be tolerated is modeled as a constrained ring cover set problem, which is a decomposition problem with exponential complexity. The face decomposition algorithm (FDA) that can tolerate one or more faults is proposed. From the results, we know that the maximum number of faults tolerated can be extended from one significantly under various network topologies.

Software reliability is defined as the probability of failure-free software operation for a specified period of time in a specified environment. Over the past 30 years, many software reliability growth models (SRGMs) have been proposed and most SRGMs assume that detected faults are immediately corrected. Actually, this assumption may not be realistic in practice. We first give a review of fault detection and correction processes in software reliability modeling. Furthermore, we show how several existing SRGMs based on NHPP models can be derived by applying the time-dependent delay function. On the other hand, it is generally observed that mutually independent software faults are on different program paths. Sometimes mutually dependent faults can be removed if and only if the leading faults were removed. Therefore, here we incorporate the ideas of fault dependency and time-dependent delay function into software reliability growth modeling. Some new SRGMs are proposed and several numerical examples are included to illustrate the results. Experimental results show that the proposed framework to incorporate both fault dependency and time-dependent delay function for SRGMs has a fairly accurate prediction capability.

We investigate an optimal resource allocation problem in modular software systems during testing phase. The main purpose is to minimize the cost of software development when the number of remaining faults and a desired reliability objective are given. An elaborated optimization algorithm based on the Lagrange multiplier method is proposed and numerical examples are illustrated. Besides, sensitivity analysis is also conducted. We analyze the sensitivity of parameters of proposed software reliability growth models and show the results in detail. In addition, we present the impact on the resource allocation problem if some parameters are either overestimated or underestimated. We can evaluate the optimal resource allocation problems for various conditions by examining the behavior of the parameters with the most significant influence. The experimental results greatly help us to identify the contributions of each selected parameter and its weight. The proposed algorithm and method can facilitate the allocation of limited testing-resource efficiently and thus the desired reliability objective during software module testing can be better achieved.

The parameters in these software reliability models are usually directly obtained from the field failure data. Due to the dynamic properties of the system and the insufficiency of the failure data, the accurate values of the parameters are hard to determine. Therefore, the sensitivity analysis is often used in this stage to deal with this problem. Sensitivity analysis provides a way to analyzing the impact of the different parameters. In order to assess the reliability of a component-based software, we propose a new approach to analyzing the reliability of the system, based on the reliabilities of the individual components and the architecture of the system. Furthermore, we present the sensitivity analysis on the reliability of a component-based software in order to determine which of the components affects the reliability of the system most. Finally, three general examples are evaluated to validate and show the effectiveness of the proposed approach.

Abstract—In this paper, we describe how several existing software reliability growth models based on Nonhomogeneous Poisson processes (NHPPs) can be comprehensively derived by applying the concept of weighted arithmetic, weighted geometric, or weighted harmonic mean. Furthermore, based on these three weighted means, we thus propose a more general NHPP model from the quasi arithmetic viewpoint. In addition to the above three means, we formulate a more general transformation that includes a parametric family of power transformations. Under this general framework, we verify the existing NHPP models and derive several new NHPP models. We show that these approaches cover a number of well-known models under different conditions. Index Terms—Software reliability growth model (SRGM), weighted arithmetic mean, weighted geometric mean, weighted harmonic mean, mean value function (MVF), power transformation, nonhomogeneous Poisson process (NHPP). 1

In this paper we propose an analytical approach for estimating the reliability of a component-based software. This methodology assumes that the software components are heterogeneous and the transfers of control between components follow a discrete time Markov process. Besides, we also formulate and. solve two resource allocation problems. Finally, we demonstrate how these analytical approaches can be employed to measure the reliability of a software system including multiple-input/multiple-output systems and distributed software systems. Experimental results show that the proposed methods can solve the testing-effort allocation problems and improve the quality and reliability of a software system.

The paper investigates the reliability of Mobile IP in a cellular system. To support the node mobility, a lot of mobility agents are deployed in the packet network of a cellular system to retain the continuous network connectivity while mobile nodes move their locations. If a mobility agent fails, the mobile nodes under its coverage area will be affected. The network connectivity of these mobile nodes will be disrupted. The main goal of this paper is to present an efficient approach to tolerating the failures of mobile agents. Once failures occur in a mobility agent, its in-progress and new-arrival workload can be redirected to other mobility agents. The mobile nodes under the mobility agent's coverage area do not lose their data executive ability. The overhead of the proposed fault-tolerant approach is also measured in terms of the performance degradation on other mobility agents by using an M/G/m/m queueing mode. The analytic results show that the performance degradation is very low if the workloads of the faulty mobility agent are not too heavy.

In this paper, based on software reliability growth models with generalized logistic testing-effort function, we study three optimal resource allocation problems in modular software systems during the testing phase: 1) minimization of the remaining faults when a fixed amount of testing-effort and a desired reliability objective are given; 2) minimization of the required amount of testing-effort when a specific number of remaining faults and a desired reliability objective are given; and 3) minimization of the cost when the number of remaining faults and a desired reliability objective are given. Several useful optimization algorithms based on the Lagrange multiplier method are proposed and numerical examples are illustrated. Our methodologies provide practical approaches to the optimization of testing-resource allocation with a reliability objective. In addition, we also introduce the testing-resource control problem and compare different resource allocation methods. Finally, we demonstrate how these analytical approaches can be employed in the integration testing. Using the proposed algorithms, project managers can allocate limited testing-resource easily and efficiently and thus achieve the highest reliability objective during software module and integration testing.

This paper investigates a SRGM (software reliability growth model) based on the NHPP (nonhomogeneous Poisson process) which incorporates a logistic testing-effort function. SRGM proposed in the literature consider the amount of testing-effort spent on software testing which can be depicted as an exponential curve, a Rayleigh curve, or a Weibull curve. However, it might not be appropriate to represent the consumption curve for testing-effort by one of those curves in some software development environments. Therefore, this paper shows that a logistic testing-effort function can be expressed as a software-development/test-effort curve and that it gives a good predictive capability based on real failure-data. Parameters are estimated, and experiments performed on actual test/debug data sets. Results from applications to a real data set are analyzed and compared with other existing models to show that the proposed model predicts better. In addition, an optimal software release policy for this model, based on cost-reliability criteria, is proposed.

This paper proposes a new scheme for constructing software reliability growth models (SRGM) based on a nonhomogeneous Poisson process (NHPP). The main focus is to provide an efficient parametric decomposition method for software reliability modeling, which considers both testing efforts and fault detection rates (FDR). In general, the software fault detection/removal mechanisms depend on previously detected/removed faults and on how testing efforts are used. From practical field studies, it is likely that we can estimate the testing efforts consumption pattern and predict the trends of FDR. A set of time-variable, testing-effort-based FDR models were developed that have the inherent flexibility of capturing a wide range of possible fault detection trends: increasing, decreasing, and constant. This scheme has a flexible structure and can model a wide spectrum of software development environments, considering various testing efforts. The paper describes the FDR, which can be obtained from historical records of previous releases or other similar software projects, and incorporates the related testing activities into this new modeling approach. The applicability of our model and the related parametric decomposition methods are demonstrated through several real data sets from various software projects. The evaluation results show that the proposed framework to incorporate testing efforts and FDR for SRGM has a fairly accurate prediction capability and it depicts the real-life situation more faithfully. This technique can be applied to wide range of software systems.

The authors show that the logistic testing effort function is practically acceptable/helpful for modeling the software reliability growth and providing a reasonable description of resource consumption. Therefore, in addition to the exponential shaped models, we integrate the logistic testing effort function into an S-shaped model for further analysis. The model is designated as the Yamada Delayed S-shaped model. A realistic failure data set is used in the experiments to demonstrate the estimation procedures and results. Furthermore, the analysis of the proposed model under an imperfect debugging environment is investigated. In fact, from these experimental results and discussions, it is apparent that the logistic testing-effort function is very suitable for making estimations of resource consumption during the software development/testing phase.

We first describe how several existing software reliability growth models based on nonhomogeneous Poisson processes (NHPPs) can be derived based on a unified theory for NHPP models. Under this general framework, we can verify existing NHPP models and derive new NHPP models. The approach covers a number of known models under different conditions. Based on these approaches, we show a method of estimating and computing software reliability growth during the operational phase. We can use this method to describe the transitions from the testing phase to operational phase. That is, we propose a method of predicting the fault detection rate to reflect changes in the user's operational environments. The proposed method offers a quantitative analysis on software failure behavior in field operation and provides useful feedback information to the development process.

We study the optimal software release problem considering cost, reliability and testing efficiency. We first propose a generalized logistic testing effort function that can be used to describe the actual consumption of resources during the software development process. We then address the problem of how to decide when to stop testing and when to release software for use. In addressing the optimal release time, we consider cost and reliability factors. Moreover, we introduce the concept of testing efficiency, and describe how reliability growth models can be adjusted to incorporate this new parameter. Theoretical results are shown and numerical illustrations are presented.

Many studies have been performed on the subject of software reliability but few have explicitly considered the impact of software testing on the reliability process. This paper presents two important issues on software reliability modeling and software reliability economics: testing effort and efficiency. First, we discuss on how to extend the logistic testing-effort function into a general form. The generalized logistic testing-effort function has the advantage of relating the work profile more directly to the natural flow of software development. Therefore, it can be used to describe the actual consumption of resources during the software development process and to obtain a conspicuous improvement in modeling testing-effort expenditures. Furthermore, we incorporate the generalized logistic testing-effort function into software reliability modeling and its fault-prediction capability is evaluated through four numerical experiments on real data. Then, we address the effects of automated techniques or tools on increasing the efficiency of software testing. New testing techniques usually increase test coverage. We propose a modified software reliability cost model to reflect these effects. From the simulation results, we obtain a powerful software economic policy which clearly indicates the benefits of applying new automated testing techniques and tools during the software development process.

Numerous stochastic models for the software failure phenomenon based on Nonhomogeneous Poisson Process (NHPP) have been proposed in the last three decades (1968-98). Although these models are quite helpful for software developers and have been widely applied at industrial organizations or research centers, we still need to do more work on examining/estimating the parameters of existing software reliability growth models (SRGMs). We investigate and account for three possible trends of software fault detection phenomena during the testing phase: increasing, decreasing and steady state. We present empirical results from quantitative studies on evaluating the fault detection process and develop a valid time-variable fault detection rate model which has the inherent flexibility of capturing a wide range of possible fault detection trends. The applicability of the proposed model and the related methods of parametric decomposition are illustrated through several real data sets from different software projects. Our evaluation results show that the analytic parametric decomposition approach for SRGM have a fairly accurate prediction capability. In addition, the testing effort control problem based on the proposed model is also demonstrated.

We investigate a software reliability growth model (SRGM) based on the Non Homogeneous Poisson Process (NHPP) which incorporates a logistic testing effort function. Software reliability growth models proposed in the literature incorporate the amount of testing effort spent on software testing which can be described by an exponential curve, a Rayleigh curve, or a Weibull curve. However it may not be reasonable to represent the consumption curve for testing effort only by an exponential, a Rayleigh or a Weibull curve in various software development environments. Therefore, we show that a logistic testing effort function can be expressed as a software development/test effort curve and give a reasonable predictive capability for the real failure data. Parameters are estimated and experiments on three actual test/debug data sets are illustrated. The results show that the software reliability growth model with logistic testing effort function can estimate the number of initial faults better than the model with Weibull type consumption curve. In addition, the optimal release policy of this model based on cost reliability criterion is discussed.