As test data widely spreading on the input domain may not thoroughly test the program's logic, in this paper, we propose an approach to generating test data widely spreading on a program's execution paths. In particular, we analyze execution paths of the program, distill constraints for executing the paths, calculate the path distance between test data according to their satisfaction for paths' constraints, and then generate test data far away from each other based on their path distance. The experimental results show that our approach significantly reduces the number of test data generated before the first fault is found.

With the popularity of Web Services and Service-Oriented Architecture (SOA), quality assurance of SOA applications, such as testing, has become a research focus. Programs implemented by the Business Process Execution Language for Web Services (WS-BPEL), which can be used to compose partner Web Services into composite Web Services, are one popular kind of SOA applications. The unique features of WS-BPEL programs bring new challenges into testing. A test case for testing a WS-BPEL program is a sequence of messages that can be received by the WS-BPEL program under test. Previous research has not studied the challenges of message-sequence generation induced by unique features of WS-BPEL as a new language. In this paper, we present a novel methodology to generate effective message sequences for testing WS-BPEL programs. To capture the order relationship in a message sequence and the constraints on correlated messages imposed by WS-BPEL's routing mechanism, we model the WS-BPEL program under test as a message-sequence graph (MSG), and generate message sequences based on MSG. We performed experiments for our method and two other techniques with six WS-BPEL programs. The results show that the message sequences generated by using our method can effectively expose faults in the WS-BPEL programs.

Due to the expensiveness of compiling and executing a large number of mutants, it is usually necessary to select a subset of mutants to substitute the whole set of generated mutants in mutation testing and analysis. Most existing research on mutant selection focused on operator-based mutant selection, i.e., determining a set of sufficient mutation operators and selecting mutants generated with only this set of mutation operators. Recently, researchers began to leverage statistical analysis to determine sufficient mutation operators using execution information of mutants. However, whether mutants selected with these sophisticated techniques are superior to randomly selected mutants remains an open question. In this paper, we empirically investigate this open question by comparing three representative operator-based mutant-selection techniques with two random techniques. Our empirical results show that operator-based mutant selection is not superior to random mutant selection. These results also indicate that random mutant selection can be a better choice and mutant selection on the basis of individual mutants is worthy of further investigation.

Techniques for test-case prioritization re-order test cases to increase their rate of fault detection. When there is a fixed time budget that does not allow the execution of all the test cases, time-aware techniques for test-case prioritization may achieve a better rate of fault detection than traditional techniques for test-case prioritization. In this paper, we propose a novel approach to time-aware test-case prioritization using integer linear programming. To evaluate our approach, we performed experiments on two subject programs involving four techniques for our approach, two techniques for an approach to time-aware test-case prioritization based on genetic algorithms, and four traditional techniques for test-case prioritization. The empirical results indicate that two of our techniques outperform all the other techniques for the two subjects under the scenarios of both general and version-specific prioritization. The empirical results also indicate that some traditional techniques with lower analysis time cost for test-case prioritization may still perform competitively when the time budget is not quite tight.

Test-case prioritization is a typical scenario of regression testing, which plays an important role in software maintenance. With the popularity of Web services, integrating Web services to build service-centric systems (SCSs) has attracted attention of many researchers and practitioners. During regression testing, as SCSs may use up constituent Web servicespsila request quotas (e.g., the upper limit of the number of requests that a user can send to a Web service during a certain time range), the quota constraint may delay fault exposure and the subsequent debugging. In this paper, we investigate quota-constrained test-case prioritization for SCSs, and propose quota-constrained strategies to maximize testing requirement coverage. We divide the testing time into time slots, and iteratively select and prioritize test cases for each time slot using integer linear programming (ILP). We performed an experimental study on our strategies together with three other strategies, and the results show that with the constraint of request quotas, our strategies can schedule test cases for execution in an order with higher effectiveness in exposing faults and achieving total and additional branch coverage.

Component Based Software Development (CBSD) is gaining popularity in recent years. In this way of software development, software components, which are typically black-box components, are intensively reused to construct new systems. To ensure the quality of software systems composed of black-box components, a primary concern is how to ensure the quality of black-box components. Thus, adequate testing of those black-box components that will be reused is a necessary step in CBSD. However, due to the unavailability of the source code of black-box components, ensuring test adequacy becomes one of the hardest issues for testing black-box components. To tackle this problem, it is a natural idea to apply mutation testing, which is a fault-based testing method used for measuring test adequacy, for component contracts, whose aim is to improve the testability of the component. Though powerful, mutation testing is usually very computation-expensive, as many mutants need to be produced and executed in mutation testing. In this paper, we propose a contract-based mutation technique for testing black-box components. Based on the discordance between contracts and specification, our approach employs a set of high level contract mutation operators. The experimental results show that these operators can greatly reduce the number of mutants compared with traditional mutation operators. At the same time, the contract-based mutation using our contract mutation operators can provide almost the same ability as that of using traditional mutation operators. Therefore, our approach can produce effective test suites to reduce the cost of testing black-box components.

Regression testing, which plays an important role in software maintenance, usually relies on test adequacy criteria to select and prioritize test cases. However, with the wide use and reuse of black-box components, such as reusable class libraries and COTS components, it is challenging to establish test adequacy criteria for testing software systems built on components whose source code is not available. Without source code or detailed documents, the misunderstanding between the system integrators and component providers has become a main factor of causing faults in component-based software. In this paper, we apply mutation on interface contracts, which can describe the rights and obligations between component users and providers, to simulate the faults that may occur in this way of software development. The mutation adequacy score for killing the mutants of interface contracts can serve as a test adequacy criterion. We performed an experimental study on three subject systems to evaluate the proposed approach together with four other existing criteria. The experimental results show that our adequacy criterion is helpful for both selecting good-quality test cases and scheduling test cases in an order of exposing faults quickly in regression testing of component-based software.

Testing plays an important role in the maintenance of component based software development. Test adequacy for component testing is one of the hardest issues for component testing. To tackle this problem, it is a natural idea to apply mutation testing, which is a fault-based testing method used for measuring test adequacy, for component contracts, whose aim is to improve the testability of the component. Though powerful, mutation testing is usually very computation-expensive, as many mutants need to be produced and executed in mutation testing. In this paper, we propose a contract-based mutation technique for testing components. Based on the discordance between contracts and specification, our approach employs a set of high level contract mutation operators. The experimental results show that these operators can greatly reduce the number of mutants compared with traditional mutation operators. At the same time, the contract-based mutation using our contract mutation operators can provide almost the same ability as that of using traditional mutation operators. Moreover, effective test suite can be produced to reduce the maintenance effort.