Flaky tests refer to test cases that exhibit inconsistent behaviors across multiple executions, potentially passing or failing unpredictably. They are frequently associated with suboptimal design practices that testers may utilize when crafting test cases, which undermine the quality of software testing. So, identifying the root causes of flaky tests is crucial for fixing them. Currently, inspired by the success of the Large Language Models (LLMs), researchers leverage the pre-trained language model to embed flaky test code as vectors and predict its root cause category based on vector similarity measures. However, such code embeddings generated by LLM mainly focus on capturing general semantic features but lack sufficient comprehension of the behavioral patterns involved in test scenarios, resulting in poor root cause identification. Test smells, which reflect poor coding practices or habits when writing test cases, provide complementary information in the root cause identification of test flakiness. Therefore, this paper proposes a root cause identification method for flaky tests based on test smells. Test smells are used to abstract and express behavioral patterns of test codes, and general semantic features extracted by vector embeddings to enhance the feature representation of flaky tests. Furthermore, to capture the complex nonlinear relationships between test smell features and code embeddings, a Feedforward Neural Network is constructed to categorize the root cause of test flakiness. To validate the effectiveness of our method, we performed evaluations on a dataset consisting of 451 Java flaky test cases. The experimental results indicate that our method achieves an F1-score of 80%, which is 7% higher than that of the baseline model that does not incorporate test smells.

In recent years, automated program repair has garnered considerable attention owing to its potential to mitigate software maintenance costs. There remain challenges for automated program repair techniques to be widely applied in practice, where too many overfitting patches are one of the issues. Automated patch correctness assessment emerges as a crucial means to enhance the viability of recommended patches in automated program repair technology. The current automated patch correctness assessment methods exhibit suboptimal performance when evaluating multiline patches compared to their effectiveness in assessing single-line patches. This disparity arises because multiline patches involve intricate contextual semantic relationships and significant program structure modifications, such as branching and looping. Those complexities render it more challenging for neural networks to capture the latent features essential for assessing correctness. To address the above problem, this paper proposes an automated patch correctness assessment method, named Peft, which fine-tunes a large language model using “golden data” that is generated based on semantic and program structural features, resulting in a patch evaluation model. Experiments were conducted on three datasets with varying proportions of multiline complex patches. The patches were sourced from real-world applications or generated by 23 APR techniques for fixing Defects4J v1.2. The results demonstrated that Peft consistently outperformed existing methods across all datasets. Notably, when evaluating datasets composed entirely of multiline complex patches, Peft significantly outperformed the open-source baseline technique Cache (accuracy: 79.2% vs. 66.1%, F1-score: 81.9% vs. 69.5%).

ABSTRACT Evolutionary test case generation plays a vital role in ensuring software quality and reliability. Since Web applications involve a large number of interactions between client and server, the dynamic evolutionary test case generation is very time‐consuming, which makes it difficult to apply in actual projects. Obviously, parallelization provides a feasible way to improve the efficiency and effectiveness of evolutionary test generation. In our previous research, the idea of parallelism has been introduced into the evolutionary test generation for Web applications. However, its performance is affected by many factors, such as migration scale, migration frequency, the number of browser processes and subpopulations, and so on. The analysis of influencing factors can guide enhancing the performance of evolutionary test generation. For this reason, this paper analyzes the factors that influence parallel evolutionary algorithms and how they affect the performance of test generation for Web applications. At the same time, different parallel evolutionary test generation methods are designed and implemented. Experiments are conducted on open‐source Web applications to generate test cases that meet the server‐side sensitive paths coverage criterion, providing guidance and suggestions for the parameter setting of parallel evolutionary test case generation for Web applications. The experimental results show that (1) compared with the global parallelization model, the evolutionary algorithm based on the parallel island model has a greater improvement in test case generation performance. In more detail, when generating test cases with the same server‐side sensitive paths coverage, the number of iterations required is reduced by 49.6%, and the time cost is reduced by 58.7%; (2) for the test case generation based on the parallel island model, if the migration scale is large, appropriately increasing the migration frequency can reduce its time cost; (3) if the number of subpopulations is fixed, appropriately increasing the number of browser processes can reduce the time cost of Web application test case evolution, but the number of browser processes should not be too large; otherwise, it may increase the time cost.

Abstract End‐to‐end (E2E) testing is a commonly used technique for web application testing. Unlike traditional unit tests, E2E tests focus on test scenarios that target the entire business, which integrate and collaborate of various components and services to make the whole application work. As a result, one drawback of E2E tests is their longer execution time, seriously affecting testing efficiency of web applications. In order to speed up the execution of E2E tests for web applications, this paper proposes a test execution optimization approach based page state reuse. Through analyzing the common operations of E2E tests, a common prefix trees is constructed to organize the same operations among test scripts in a test suite of web applications. Under the guidance of the prefix tree, the optimal reusable state is identified and duplicated to maximize the utilization of page states triggered by the same operations, thereby reducing the overall execution time of the test suite. Besides, to realize page state reuse automatically, we design a browser process replication strategy, which implement querying the active page and duplicating the web page. To verify the effectiveness of our method, experiments and evaluations were conducted on 347 E2E tests from eight open‐source web applications, and the results showed that our approach reduced the E2E testing execution time for web applications by 52%–68%.

Flaky tests can non-deterministically pass or fail on the same version of code. Their occurrence prevents using test results to determine if there are bugs in the program, thereby reducing the credibility of software testing. Concurrency is a main cause of flaky tests, where different concurrent thread interleaving may lead to uncertain program execution results. Essentially, inconsistent test execution results are more likely to occur when threads with data dependencies are interleaved. But the existing work scheduled threads randomly with no guide during flaky test detection, resulting in a vast interleaving space and low efficiency. To address this issue, we propose a data conflict-guided flaky test detection approach, which prioritizes threads with data dependency for scheduling to try to alter the test results, thereby detecting flaky tests more effectively. In more detail, the dynamic execution trace of test cases is tracked in multithreaded programs. Then, by analyzing read-write operations in the trace, data conflicts are identified. On this basis, a Bayesian Network is put forward to evaluate the degree of impact of data conflicts on test flakiness, which implies potential flaky risk. That is, the greater the flaky risk, the more likely the data conflict is to cause a flaky test. So, the scheduling strategy prioritizes threads with data conflict pairs that have a significant impact on test flakiness, to trigger test flakiness as soon as possible. To verify the effectiveness of our approach, we conduct experiments on multithreaded Java programs. The experimental results show that our approach can discover the flakiness of tests within an average of 118 seconds, and the accuracy in detecting flaky tests can reach 75 %.

Model-based testing provides an effective means for ensuring the quality of Android apps. Nevertheless, existing models that focus on event sequences and abstract them into Finite State Machines (FSMs) may lack precision and determinism because of the different data values of events that can result in various states of Android applications. To address this issue, a novel model based on Extended Finite State Machines (EFSMs) for Android apps is proposed in this paper. The approach leverages machine learning to infer data constraints on events and annotates them on state transitions, leading to a more precise and deterministic model. Additionally, a state abstraction strategy is presented to further refine the model. Besides, test diversity plays a vital role in enhancing test suite effectiveness. To achieve high coverage and fault detection, test cases are generated from the EFSM model with the help of a Genetic Algorithm (GA), guided by test diversity. To evaluate the effectiveness of our approach, this paper carries out experiments on 93 open-source apps. The results show that our approach performs better in code coverage and crash detection than the existing open-source model-based testing tools. Particularly, the 19 unique crashes that involve complex data constraints are detected by our approach.

Tests may pass or fail under the same conditions. These tests are commonly known as flaky tests. In Android applications, the primary reason for flaky tests is attributed to its event-driven programming paradigm and multi-threading concurrency mechanism. It may activate an unexpected event order when a test is executed, causing test flakiness. The later the execution of asynchronous events, the more likely it is to result in test flakiness. Inspired by this deduction, this paper puts forward a flaky test detection method for concurrent Android applications based on adaptive latest position execution. In more detail, the latest execution positions of each asynchronous event are identified by analyzing the sequential dependencies between events. On this basis, the asynchronous event is scheduled at the corresponding position, thereby trying to change the test results and detecting flaky tests. To validate the effectiveness and efficiency of our approach, a series of experiments are conducted on 16 known flaky test cases across 7 Android applications. The experimental results show that compared with the state-of-the-art tool FlakeScanner, the flaky test detection rate of our approach improves by 18.75%.

In response to the inefficiency of traditional testing and limitations of existing platforms for chip verification, this article introduces a method for automatically generating verification instruction sets for DSP chips. It aims to address scalability, portability, and ensure comprehensive coverage of chip functionalities. The article defines instruction templates based on DSP chip instructions' content and format address, serving as a foundation for diverse verification instruction sets. Constraints and coverage evaluation standards are formulated to guarantee full chip function coverage and high code coverage. An algorithm facilitates automatic generation. A simulation verification platform is designed to validate the method, seamlessly integrating with existing infrastructure. It enables automatic generation of module-specific verification instruction sets and comprehensive verification of the ALU floating-point arithmetic unit. This methodology successfully handles verification tasks with millions of instructions. Experimental results demonstrate full functional coverage and 100%operand and instruction option coverage, surpassing traditional approaches. Automatic generation enhances the verification of DSP chip components, improving overall quality and reliability.

Analog circuit is an important component of modern electronic systems. However, the soft fault diagnosis of analog circuits is challenging due to their large parameter variability and complex internal structure. So, this paper proposes an automatic fault diagnosis method based on Multi-Scale 1D Convolutional Neural Network (MS-1D-CNN) for analog circuits. Considering that faults may disappear or weaken during the propagation process and cannot be manifested in the output signals, the fault diagnosis model is trained and constructed based on an optimum set of test points with the maximum degree of fault isolation and least test points. Furthermore, because the data at different test points and time periods have different influences on fault diagnosis, a mixed attention mechanism combining both channel and spatial attention is adopted to extract more critical information in the fault diagnosis model, achieving a more accurate soft fault diagnosis for analog circuits. Experiments are conducted on four widely used benchmark circuits. The results show that our method has higher accuracy of fault diagnosis than the existing methods, and the fault diagnosis model based on multi-test points data has higher accuracy than that solely based on the output signals for analog circuits.

Test case prioritization based on reinforcement learning has been seen as a promising way to achieve continuous integration testing. Agent and reward function are two crucial components of reinforcement learning. During the process of reinforcement learning in continuous integration test case prioritization, the agent decides on the execution order of test cases (actions) for the next integration testing (environment) based on the corresponding test case features (states), aiming to detect errors early by maximizing the reward. Furthermore, having more test case features allows the agent to perceive the environment better, but it also increases computation consumption and brings convergence problems to learning. In this paper, we first propose a multi-feature environment perception for continuous integration test case prioritization. It introduces multiple features based on test case history execution information to solve the agent’s limitation in obtaining environmental information. Additionally, we propose a self-attention agent network structure, which captures relationships between multiple features to prevent the convergence problem of reinforcement learning. An extensive experimental and analytical study was conducted with 15 existing reward functions on 14 industrial data sets. The results show that (1) the proposed multiple features can help the agent to perceive environmental information more comprehensively, and (2) the proposed self-attention agent can process environmental information better to achieve more effective test case prioritization in continuous integration testing.

In the continuous integration, test case prioritization can effectively alleviate the resource-intensive problems associated with frequent integration commits. Test case prioritization in continuous integration is a sequential decision problem from which reinforcement learning is applied and can effectively adapt and learn from a changing environment. However, continuous integration testing brings new problems of sparse rewards to reinforcement learning because of frequent integration with low test failure and this problem can be addressed by increasing the number of rewarded test cases. In this paper, we propose a reinforcement learning reward object selection strategy based on Test Case Synchronization and Diversity (TCSD) that rewards failed test cases and with an additional selection of passed test cases with potential failure ability. The experiments on six real-world industrial data sets show that TCSD improves the learning efficiency and fault detection ability of reinforcement learning 6.35% in average NAPFD compared with the traditional strategies.

By scheduling algorithms in the low-level algorithm library, the hyper-heuristic algorithm can help to effectively select an appropriate method to deal with hard computational search problems. The hyper-heuristic algorithm usually includes a high-level scheduling layer and a low-level algorithm layer. The high-level strategy layer selects the algorithm for the next scheduling by evaluating the execution effect of the different algorithms in the low-level layer, while the low-level layer includes a variety of different heuristic algorithms which called algorithm library. The concrete hyper-heuristic framework for multi-objective test case prioritization was presented where the 18 multi-objective algorithms were formed in the low-level library. It has been gradually realized that a hybrid algorithm by combining single objective algorithm and multi-objective optimization algorithm is better than the individual. This paper explores the influence of the construction pattern of algorithm library for the hyper-heuristic algorithm by constructing the fusion pattern of different types of algorithms.

Learning agent plays significant role in the hyper-heuristic framework for test case prioritization, where an evaluation strategy is applied to evaluate the execution results produced by the current heuristic algorithm and select the most appropriate heuristic algorithm for the next generation. Hierarchical Distribution (HD) is used as evaluation strategy based on the dominance relationship between the individuals from the present and last generations. In addition to the distribution of the solution set, a good convergence towards the optimal Pareto front is often desired. In this paper, the convergence ability of the individuals is further considered in the design of the evaluation strategy for the learning agent, in which Pareto Dominance and Convergence Information are adopted. Three evaluation strategies are proposed and empirically studied, and the experimental results show that the hyper-heuristic algorithms with the proposed evaluation strategies are more effective and efficient for test case prioritization.

Testing multi-thread programs becomes extremely difficult because thread interleavings are uncertain, which may cause a program getting different results in each execution. Thus, Thread Scheduling Sequence (TSS) is a crucial factor in multi-thread program testing. A good TSS can obtain better testing efficiency and save the testing cost especially with the increase of thread numbers. Focusing on the above problem, in this paper, we discuss a kind of approach that can efficiently generate TSS based on the concurrent coverage criteria. First, we give a definition of Synchronization Pair (SP) as well as all Synchronization Pairs Coverage (ASPC) criterion. Then, we introduce the Synchronization Pair Thread Graph (SPTG) to describe the relationships between SPs and threads. Moreover, this paper presents a TSS generation method based on the ASPC according to SPTG. Finally, TSSs automatic generation experiments are conducted on six multi-thread programs in Java Library with the help of Java Path Finder (JPF) tool. The experimental results illustrate that our method not only generates TSSs to cover all SPs but also requires less state number, transition number as well as TSS number when satisfying ASPC, compared with other three widely used TSS generation methods. As a result, it is clear that the efficiency of TSS generation is obviously improved.

Spiking Neuron Network (SNN) uses spike sequence for data processing, so it has an excellent characteristic of low power consumption. However, due to the immaturity of learning algorithm, the multiplayer network training has difficulty in convergence. Utilizing the mature learning algorithm and fast training speed of the back-propagation network, this paper proposes a method to converse the Convolutional Neural Network (CNN) to the SNN. First, the adjustment strategy for CNN is introduced. Then after training, the weight parameters in the model are extracted, which is the corresponding synaptic weight in the layer of the SNN. Finally, a new threshold-setting algorithm based on feedback is proposed to solve the critical problem of the threshold setting of neurons in the SNN. We evaluate our method on the CIFAR-10 datasets released by Hinton’s team. The experimental results show that the image classification accuracy of the SNN is more than 98% of that of CNN, and the theoretical value of power consumption per second is 3.9[Formula: see text]mW.

In this article, test diversity has been suggested to be a valid way to improve test suite effectiveness. Extended finite state machine (EFSM) is a widely used formal model, but little attention is paid on the test suite generation with more diversity. EFSM test suite generation involves test paths generation and test data generation. Considering the discrepancy between test paths has a more crucial impact on the diversity of test suite, compared with the difference between test data, this article, therefore, mainly concerns the test paths generation with more diversity for EFSM models. Hence, the factors that influence the discrepancy between test paths are investigated. Then based on these factors, an integrated distance metric is designed to evaluate the dissimilarity between test paths, and a diversity measurement for EFSM test suite is presented. Furthermore, a diversity-oriented test suite generation (DOTSG) method is proposed where a dissimilarity-based fitness function and diversity-oriented update strategy are adopted in traditional coverage-oriented EFSM test suite generation (COTSG) by genetic algorithm. The experimental results show that, compared to COTSG, our DOTSG can not only generate more diverse test suite to satisfy a certain coverage criteria, improving the fault detection capability of the test suite, but also decrease the evolution time cost and the size of test suite generated.

Continuous integration refers to the practice of merging the working copies of all developers into the mainline frequently. Regression testing for each mergence is characterized by continually changing test suite, limited execution time, and fast feedback, which demands new test optimization techniques. Reinforcement learning is introduced for test case prioritization to save computing resources in continuous integration environment, where a reasonable reward function is highly important for learning strategy, since the process of reinforcement learning is a reward-guided behavior. In this paper, APHFW, a novel reward function is proposed by using partial historical information of test cases effectively for fast feedback and cost reduction. The experiments are based on three open-source data sets, and the results show that the proposed reward function is more cost-effect than other reinforcement learning rewards in continuous integration environment.

Currently, more than 90% web applications are potentially vulnerable to attacks from both the client side and server side. Test case generation plays a crucial role in testing web applications, where most existing studies focus on test case generation either from client-side or from server-side to detect vulnerabilities, regardless of the interactions between client and server. Consequently, it is difficult for those test cases to discover certain faults which involve both client and server. In this paper, the server-side sensitive paths are considered as vulnerable code paths due to insufficient or erroneous filtering mechanisms. An evolutionary testing approach based on the memetic algorithm is proposed to connect the server-side and client-side, in which test cases are generated from the client-side behavior model, while guided by the coverage of sensitive paths from server-side. The experiments are conducted on four open source web applications, and the results demonstrate that our approach can generate test cases from the client-side behavior model that can cover the server-side sensitive paths, on which the vulnerabilities can be detected more effectively.

Test code generated by using the page object design pattern during web testing is easy to maintain.Page clustering is an essential stage of the page object approach.However, existing methods only consider the DOM structure in page clustering, which leads to inaccuracy when generating page objects.A state with the same DOM structure may result in an entirely different migration.The method of considering only the DOM structure cannot accurately generate page object classes.In order to improve the accuracy of page object generation, this paper not only considers DOM structure information but also considers CSS styles and the attributes of DOM elements in page clustering.Based on the experimental evaluation results, our method can automatically generate page objects that cover most of the application functions, which is more effective for the creation and maintenance of web test cases.

Program faults are unavoidable phenomena in the software development. The application of mutation and fault localization techniques is effective in automated program repair, but suffers the inevitable high execution cost as the number of mutants increases exponentially for large industrial programs. The combination with fault localization techniques can reduce the cost by mutating the statements with high suspicious values first. However, the accuracy of fault localization techniques has not been good enough for real applications, and some faults may occur on a statement which is related to the statement with a high suspicious value rather than itself. Therefore, the greedy strategy used currently may not be effective, resulting in inefficient repairs. Finding a mutant as a correct patch should be regarded as a continuous process of searching the global solutions. In this paper, we proposed the search-based automated program repair using mutation and fault localization, which not only takes advantage of fault localization but also overcomes the disadvantage of the greedy strategy used in the mutation generation. The initial population of the search algorithm is constructed by the mutants generated from the statements with high suspicious values using the fault localization, which is a set of rough solutions. A hybrid-crossover operator is then designed where the fixed position crossover operator is used to converge to the global optimal solutions and the random position crossover operator is used to explore the entire search space faster, respectively. The experimental results on the Siemens suite indicate that the proposed approach can improve the efficiency with the same effectiveness compared to the exhaustive approach, and show that the non-random initial population method and the hybrid-crossover strategy can improve the efficiency of the search process.

Most of web applications models focus on sequencing of events, where the ignored parameters or DOM elements changes and the relationship between the execution conditions and web states are crucial for analyzing and testing the behavior of client-side of client-server web applications. In this paper, we first define a novel trace, which can represent dynamic behaviors of web applications more accurately. Then an EFSM-oriented minimal traces set generation approach is proposed for modelling web applications. In order to ensure the integrity of the EFSM model and improve the effectiveness of the modelling process, three adequacy criteria with respect to all events, JS branches and DOM structures, are applied to compensate the traces and to guide the minimal traces set generation by greedy algorithm. Finally, the minimal traces set is abstracted into an EFSM as the behavior model for web applications. We implement a prototype tool for the proposed approach and empirically evaluate that the minimal traces set generation approach based on two web applications. The results show that the traces generated by the approach is effective and the all JavaScript branches coverage criteria is most appropriate to select the minimal traces set used for modelling.

Test cases generation is a principal process in web application testing.Most existing methods generate test cases for improving test efficiency mainly from the aspects like minimizing the test case suite, increasing the code coverage, and so on.However, similar with traditional software having important functions, classes or modules, some web states are more vital than others in web applications.It can be thought that those vital web states relatively have higher influence on the performance of web application.So, they should be given more attention in test case generation.In more detail, the importance of web states can be measured from its page contents or topological structures.Meanwhile, as we known Model-based Testing is a kind of widely used approach in automatic test case generation.Therefore in this paper we propose an EFSM based test case generation method considering the importance of web states for web applications.The experimental results show that our methods can deterministically enhance the testing efficiency of web application.

Abstract Test case prioritization (TCP), which aims to find the optimal test case execution sequences for specific testing objects, has been widely used in regression testing. A wide variety of search methodologies and algorithms have been proposed to optimize test case execution sequences, namely, search‐based TCP. However, different algorithms perform differently and have different implementation costs and specific situations where an algorithm usually performs with high effectiveness and efficiency. When facing a new testing scenario, it is actually difficult to decide which algorithm is suitable. In this paper, to address the algorithm selection problem for different test scenarios, a more generally applicable algorithm based on a hyperheuristic strategy is proposed for search‐based TCP. This includes a range of multiobjective algorithms with a variety of crossover strategies and a learning agent strategy to evaluate and select the appropriate algorithm execution sequence dynamically for different scenarios. The concrete hyperheuristic framework for multiobjective TCP is presented with an algorithm's repository in the low level and the learning agent strategy in the higher level. Experiments show that the proposed learning agent strategy can accurately evaluate algorithms in multiobjective problems and select the appropriate algorithm in each iteration.

The commonly-used software fault localization approaches mainly utilize test coverage information and test cases execution results to calculate the suspiciousness of each program entity to identify the location of faults, namely spectrum based software fault localization (SBFL). It had been argued that such techniques are not helpful in real debugging process, since the low accuracy of localization and few information provided to programmers. In this paper we consider the combination of statement based fault classification with the SBFL, aiming at increasing accuracy of fault localization and provide additional possible fault information to programmers. An improved technique, fault classification oriented SBFL (FC-SBFL), is proposed in this paper, in which the suspiciousness value is adjusted dynamically based on the probability of statement being faulty. Experimental results on real application programs show that FC-SBFL is more effective than SBFL to locate faults, and studies with Tarantula and OP2 show that more than 75% faults have been identified in a better effectiveness.

Mutation Based Fault Localization(MBFL) is a fault localization technique based on mutation analysis, which precisely identifies the location of fault but incurs a high execution cost, since it needs to execute the test suite on a large amount of mutants. Reduction strategies proposed are usually regarding selecting mutation operators or sampling mutants directly, meanwhile at the cost of losing precision of fault localization. This paper proposes a Statement-Oriented Mutant Reduction strategy (SOME), which selects a proportion of mutants at the statement level, specifically, the statements covered by failed tests. SOME keeps the advantage of using whole types of mutation operators, and further considers the increase of mutants' diversity to avoid the precision loss of fault localization. Empirical studies are conducted on 112 faulty versions from 7 benchmark programs, and the results indicate that SOME can reduce 73.51%-79.98% mutation execution cost while keeping almost the same fault location precision as the original MBFL without reduction..

Test data generation is a process of creating program inputs that satisfy specific testing criteria. Many works have been focused on test generation with respect to numeric and string data. Dynamic data structures, such as trees and linked lists, have been widely used in modern programming, but on which there are few studies presented. In general, generating a dynamic data structure is associated with a proper shape and valid values generation. It would be difficult to generate such dynamic data structures, as both shapes and values are necessary to be valid simultaneously. This paper focuses on binary tree structures and proposes a novel test generation approach that combines search based testing with constraint solving techniques. The approach creates the shapes of binary tree structures by using GA, and generates the values in their data fields by using constraint solving techniques. The experimental results show that the presented approach is promising and effective. Moreover, the studies investigate factors affecting the performance of the approach, and arrive at a conclusion that the test generation cost is cubic growing as the number of pointer constraints increases.

Abstract Domain testing is designed to detect domain errors that result from a small boundary shift in a path domain. Although many researchers have studied domain testing, automatic domain test data generation for string predicates has seldom been explored. This paper presents a novel approach for the automatic generation ofON–OFFtest points for string predicate borders, and describes a corresponding test data generator. Our empirical work is conducted on a set of programs with string predicates, where extensive trials have been done for each string predicate, and the results are analysed using the SPSS tool. Conclusions are drawn that: (i) the approach is promising and effective; (ii) there is a strong linear relationship between the performance of the test generator and the length of target string in the predicate tested; and (iii) initial inputs, no shorter than the target string and with characters generated randomly, may enhance the performance in the test data generation for string predicates. Copyright © 2009 John Wiley & Sons, Ltd.

A key issue in black-box testing is how to select adequate test cases from input domain on the basis of specification. However, for some kinds of software, developing test cases from output domain is more suitable than from input domain. In this paper, we present a novel approach to automatically generate test cases from output domain. A model is created via neural network to take as a function substitute for the software under test, and then on the basis of the created function model, for given outputs we employ an improved genetic algorithm to find the corresponding inputs, so that the automation of test cases generation from output domain is completed. In order to investigate the effectiveness of the approach, a number of experiments have been conducted on two different software programs under test. Experimental results show that this approach is promising and effective.

Nowadays, many test data generation approaches are employed on basic numerical types such as integer and real data. However, in real practice, pointers and dynamic data structures are so widely used that most recent test data generation approaches are restricted in application. This paper proposes a path-oriented test data generation approach specifically for dynamic pointer data. Firstly, a least restrictive shape involved in input structure is created, which meets pointer constraints for a given path. Secondly, the value of data field in the created shape is determined. The experiment results show that our approach is effective and practicable in test generation for dynamic pointer data.

Domain testing is a well-known software testing technique. Although research tasks have been initiated in domain testing, automatic test data generation based on character string predicates has not yet been reported. This paper presents a novel approach to automatically generate ON-OFF test points for character string predicate borders associated with program paths, and describes a corresponding test data generator control flow based testing Slices with respect to predicates on paths are constructed to calculate the current values of variables in the predicates via program slicing techniques. Each character element of variables in a character string predicate is dynamically determined in turn by function minimization so that the ON-OFF test points for the predicate border can be automatically generated. The preliminary experimental results show that this approach is promising and effective.

Partition testing is a well-known software testing technique. This paper shows that partition testing strategies are relatively ineffective in detecting faults related to small shifts in input domain boundary. We present an innovative software testing approach based on input domain analysis of specifications and programs, and propose the principle and procedure of boundary test case selection in functional domain and operational domain. The differences of the two domains are examined by analyzing the set of their boundary test cases. To automatically determine the operational domain of a program, the ADSOD system is prototyped. The system supports not only the determination of input domain of integer and real data types, but also non-numeric data types such as characters and enumerated types. It consists of several modules in finding illegal values of input variables with respect to specific expressions. We apply the new testing approach to some example studies. A preliminary evaluation on fault detection effectiveness and code coverage illustrates that the approach is highly effective in detecting faults due to small shifts in the input domain boundary, and is more economical in test case generation than the partition testing strategies.