Many interesting questions about the software quality of a code base can only be answered adequately if fine-grained information about the evolution of quality metrics over time and the contributions of individual developers is known. We present an approach for tracking static analysis violations (which are often indicative of defects) over the revision history of a program, and for precisely attributing the introduction and elimination of these violations to individual developers. As one application, we demonstrate how this information can be used to compute ``fingerprints'' of developers that reflect which kinds of violations they tend to introduce or to fix. We have performed an experimental study on several large open-source projects written in different languages, providing evidence that these fingerprints are well-defined and capture characteristic information about the coding habits of individual developers.

Specifications mined from execution traces are largely used to support testing and analysis of software applications with little runtime variability. However, when models are mined from applications that evolve at runtime, the resulting models become quickly obsolete, and thus of little support for any testing and analysis activity. To cope with such systems, mined specifications must be consistently updated every time the software changes. In principle, models can be periodically mined from scratch, but in many cases this solution is too expensive or even impossible. In this paper we describe Revolution, an approach for the automatic evolution of specifications mined by applying state abstraction techniques. Revolution produces models that are continuously updated and thus remain aligned with the actual implementation. Empirical results show that Revolution can suitably address run-time evolving applications.

Evolutionary structural testing has been researched and promising results have been presented. However, it has hardly been applied to real-world complex systems and as such, little is known about the scalability, applicability and acceptability of it in an industrial setting. The European project EvoTest (IST-33472) team has been working from 2006 till 2009 to improve this situation and this paper informs about the results. We start with an overview of tools and techniques which we have developed for automated evolutionary structural testing. Subsequently, we describe the empirical setup used to study the applicability of evolutionary structural testing in industry through two case studies. The test objects used for the studies are selected functions (handwritten and generated) from production systems at Daimler and Berner & Mattner Systemtechnik (BMS) like, for example, Rear Window Defroster, Global Powertrain Engine Controller, Window Lift Control System, etc. The results of the case studies are described and research questions are assessed based on the obtained results. In summary, the results indicate that evolutionary structural testing in an industrial setting is worthwhile and profitable. Hardly any detailed knowledge of evolutionary computation is required to search for interesting test data. The case studies also research the benefits of using techniques like automated parameter tuning and search space smoothing.

We present an algorithm for constructing fitness functions that improve the efficiency of search-based testing when trying to generate branch adequate test data. The algorithm combines symbolic information with dynamic analysis and has two key advantages: It does not require any change in the underlying test data generation technique and it avoids many problems traditionally associated with symbolic execution, in particular the presence of loops. We have evaluated the algorithm on industrial closed source and open source systems using both local and global search-based testing techniques, demonstrating that both are statistically significantly more efficient using our approach. The test for significance was done using a one-sided, paired Wilcoxon signed rank test. On average, the local search requires 23.41% and the global search 7.78% fewer fitness evaluations when using a symbolic execution based fitness function generated by the algorithm.

The current Web applications are in continuous evolution to provide new and more complex functionalities, which can improve the user experience by means of adaptivity and dynamic changes. Since testing is the most frequently used technique to evaluate the quality of software applications in industry, novel testing approaches will be necessary to evaluate the quality of future (and more complex) web applications. In this paper, we investigate the testing challenges of future web applications and propose a testing methodology that addresses these challenges by the integration of search-based testing, model-based testing, oracle learning, concurrency testing, combinatorial testing, regression testing, and coverage analysis. This paper also presents a testing metamodel that states testing concepts and their relationships, which are used as the theoretical basis of the proposed testing methodology.

Evolutionary testing has been researched and promising results have been presented. However, evolutionary testing has remained predominately a research-based activity not practiced within industry. Although attempts have been made, such as Daimler's Evolutionary Structural Test (EST) prototype, until now, no such tool has been suitable for industrial adoption. The European project EvoTest (IST-33472) team has been working from 2006 till 2009 to improve this situation. This paper describes the final version of the Evolutionary Testing Framework (ETF) resulting from the EvoTest project. In specific we will present the EvoTest Structural Testing tool for fully automatic structural testing that has been demonstrated to be suitable within an industrial setting. The paper concentrates on how to use it and interpret the results. The paper starts with introducing the concepts of Evolutionary Testing in general and Structural Testing in specific. Subsequently, the ETF and the EvoTest Structural Testing tool built on-top of it will be described. We will concentrate on the usage, the architecture, and remaining limitations of the tool. The paper concludes describing the results of using the EvoTest Structural Testing tool in practice on real-world systems in an industrial setting.

This paper presents an automatic trace-based unit testing approach to test object-oriented programs. Most automated testing tools test a class C by testing each of its methods in isolation. Such an approach works poorly if specifications are only partial, which is usually the case in practice. In contrast, our approach generates sequences of calls to the methods of C that are checked on-the-fly. This is more interactive, and has the side effect that methods are checking each other. Although simple, it seems to work quite well, even when specifications are only partially provided. We implement the approach in a tool called T2. It targets Java. It can test internal errors, Hoare triple specifications, class invariant, and even temporal properties. Furthermore, T2 accepts ’in-code’ specifications, these are specifications written in the specified class itself, and are written in plain Java; hence reducing the cost usually needed to maintain specifications to minimum.

We present techniques for representing typed abstract syntax trees in the presence of observable recursive structures. The need for this arose from the desire to cope with left-recursion in combinator based parsers. The techniques employed can be used in a much wider setting however, since it enables the inspection and transformation of any program structure, which contains internal references. The hard part of the work is to perform such analyses and transformations in a setting in which the Haskell type checker is still able to statically check the correctness of the program representations, and hence the type correctness of the transformed program.

A permutation phrase is a sequence of elements (possibly of different types) in which each element occurs exactly once and the order is irrelevant. Some of the permutable elements may be optional. We show how to extend a parser combinator library with support for parsing such free-order constructs. A user of the library can easily write parsers for permutation phrases and does not need to care about checking and reordering the recognized elements. Applications include the generation of parsers for attributes of XML tags and Haskell's record syntax.

Even when programming in a statically typed language we every now and then encounter statically untypable values; such values result from interpreting values or from communicating with the outside world. To cope with this problem most languages include some form of dynamic types. It may be that the core language has been explicitly extended with such a type, or that one is allowed to live dangerously by using functions like unsafeCoerce. We show how, by a careful use of existentially and universally quantified types, one may achievem the same effect, without extending the language with new or unsafe features. The techniques explained are universally applicable, provided the core language is expressive enough; this is the case for the common implementations of Haskell. The techniques are used in the description of a type checking compiler that, starting from an expression term, constructs a typed function representing the semantics of that expression. In this function the overhead associated with the type checking is only once being paid for; in this sense we have thus achieved static type checking.