During the development of electronic control units (ECU) in domains like the automotive industry, tests are performed on various test platforms, such as model-in-the-loop, software-in-the-loop, processor-in-the-loop, and hardware-in-the-loop platforms in order to find faults in early development stages. Test cases must be specified to verify the properties demanded of the system on these test platforms. This is an expensive and non-trivial task. Evolutionary black-box testing, a recent approach to automating the creation of interesting test cases, can solve this task completely automatically.

Evolutionary functional testing is an approach to automatically generating test cases that violate a certain functional system requirement. This paper proposes an approach suitable for efficiently testing complex continuous control systems using an evolutionary testing framework that has been optimized for deployment in the industrial domain. Features of this test environment include the ability of automatically generating and optimizing continuous test data sets. The efficiency of the method is assessed by performing an experimental case study and by comparison with random testing. Results of this work indicate high usability and efficiency of the proposed method.

Evolutionary structural testing is an approach to automatically generating test cases that achieve high structural code coverage. It typically uses genetic algorithms (GAs) to search for relevant test cases. In recent investigations particle swarm optimization (PSO), an alternative search technique, often outperformed GAs when applied to various problems. This raises the question of how PSO competes with GAs in the context of evolutionary structural testing. In order to contribute to an answer to this question, we performed experiments with 25 small artificial test objects and 13 more complex industrial test objects taken from various development projects. The results show that PSO outperforms GAs for most code elements to be covered in terms of effectiveness and efficiency.

Automating the generation of object-oriented unit tests is a challenging task. This is mainly due to the complexity and peculiarities that the principles of object-orientation imply. One of these principles is the encapsulation of class members which prevents non-public methods and attributes of the class under test from being freely accessed.

Timing constraints in embedded systems must be satisfied so that real-time embedded systems work properly and safely. Execution time testing involves finding the best and worst case execution times. Evolutionary testing is used to dynamically search for the extreme execution times. During the evolutionary search, some parts of the source code are never accessed. Moreover, it turns out that the search delivers different extreme execution times in a high number of generations. We propose a new approach which makes use of seeding the evolutionary algorithm with test data achieved a high structural coverage. This new method leads to raise the confidence in the results and to gain in efficiency in terms of number of generations needed.

Evolutionary algorithms have been successfully applied in the area of software testing. However, previous approaches in the area of object-oriented testing are limited in terms of test case feasibility due to call dependences and runtime exceptions. In this paper, we present a search-based approach to automatically generating test cases for object-oriented software. It relies on a tree-based representation of method call sequences. Strongly-typed genetic programming is employed to generate method call trees which respect the call dependences among the methods. We apply a new kind of distance-based fitness function that accounts for runtime exceptions. In a case study, the approach outperformed random testing in terms of achieved coverage and it produced test cases achieving full branch coverage for a test object that makes ample use of explicit runtime exceptions.

Embedded systems are often used in a safety-critical context, e.g. in airborne or vehicle systems. Typically, timing constraints must be satisfied so that real-time embedded systems work properly and safely. Execution time testing involves finding the best and worst case execution times to determine if timing constraints are respected. Evolutionary real-time testing (ERTT) is used to dynamically search for the extreme execution times. It can be shown that ERTT outperforms the traditional methods based on static analysis. However, during the evolutionary search, some parts of the source code are never accessed. Moreover, it turns out that ERTT delivers different extreme execution times in a high number of generations for the same test object, the results are neither reliable nor efficient. We propose a new approach to ERTT which makes use of seeding the evolutionary algorithm with test data achieving a high structural coverage. Using such test data ensures a comprehensive exploration of the search space and leads to rise the confidence in the results. We present also another improvement method based on restricting the range of the input variables in the initial population in order to reduce the search space. Experiments with these approaches demonstrate an increase of reliability in terms of constant extreme execution times and a gain in efficiency in terms of number of generations needed.

Evolutionary algorithms have successfully been applied to software testing. Not only approaches that search for numeric test data for procedural test objects have been investigated, but also techniques for automatically generating test programs that represent object-oriented unit test cases. Compared to numeric test data, test programs optimized for object-oriented unit testing are more complex. Method call sequences that realize interesting test scenarios must be evolved. An arbitrary method call sequence is not necessarily feasible due to call dependences which exist among the methods that potentially appear in a method call sequence. The approach presented in this paper relies on a tree-based representation of method call sequences by which sequence feasibility is preserved throughout the entire search process. In contrast to other approaches in this area, neither repair of individuals nor penalty mechanisms are required. Strongly-typed genetic programming is employed to generate method call trees. In order to deal with runtime exceptions, we use an extended distance-based fitness function. We performed experiments with four test objects. The initial results are promising: high code coverages were achieved completely automatically for all of the test objects.

White-box testing is an important method for the early detection of errors during software development. In this process test case generation plays a crucial role, defining appropriate and error-sensitive test data. The evolutionary white-box testing is a promising approach for the complete automation of structure-oriented test case generation. Here, test case generation can be completely automated with the help of evolutionary algorithms. However, problem cases exist in which the evolutionary test is not able to find valid test data. Thus, in the case of not achieving a test goal, it is not known whether this is due to non-executable program code or a problem case. This paper will investigate how successfully a software measure can support an evolutionary white-box test if the measure can predict the test effort. Hence, the termination criteria of evolutionary white-box testing can be adapted to test goals with problem cases in such a way that problematic test goals are either excluded from the test in advance or can be covered due to an adequate termination criteria according to a software measure. This could lead to an increase in efficiency and effectiveness of evolutionary white-box testing.

As the paradigm of object orientation becomes more and more important for modern IT development projects, the demand for an automated test case generation to dynamically test object-oriented software increases. While search-based test case generation strategies, such as evolutionary testing, are well researched for procedural software, relatively little research has been done in the area of evolutionary object-oriented software testing.This paper presents an approach with which to apply evolutionary algorithms for the automatic generation of test cases for the white-box testing of object-oriented software. Test cases for testing object-oriented software include test programs which create and manipulate objects in order to achieve a certain test goal. Strategies for the encoding of test cases to evolvable data structures as well as ideas about how the objective functions could allow for a sophisticated evaluation are proposed. It is expected that the ideas herein can be adapted for other unit testing methods as well.The approach has been implemented by a prototype for empirical validation. In experiments with this prototype, evolutionary testing outperformed random testing. Evolutionary algorithms could be successfully applied for the white-box testing of object-oriented software.