Software testing began as an empirical activity, and remains part of engineering practice without a widely accepted theoretical foundation. The overwhelming majority of test methods are designed to find software errors, termed faults, in program source code, but not to assess software operational quality. To go beyond fault-seeking requires a theory that relates static program properties to executions. In the 1970s and 1980s, Gerhart, Howden, and others developed a sound functional theory of program testing. Then Duran and others used this theory to precisely define the notions of random testing and operational reliability. In the Gerhart-Howden-Duran theory, a program's behavior is a pure input-output mapping. This paper extends the theory to include persistent state, by adding a state space to the input space, and a state mapping to a program's output mapping. The extended theory is significantly different because test states, unlike inputs, cannot be chosen arbitrarily. The theory is used to analyze state-based testing methods, to examine the practicality of reliability assessment, and to suggest experiments that would increase understanding of the statistical properties of software.

This paper examines three ideas: First, the traditional relationship between a science, the mathematics it uses and the engineering based on it. Second, the nature of (software) computer science, which may not be a science at all, and its unusual use of mathematics. And finally, the nature of software engineering, its relationship with computer science, and its use of mathematics called ‘formal methods’. These three ideas turn on the first of them, since the scientific world view seems natural for the study of computing. The paper's thesis is that while software touches science in many ways, it does not itself have a significant scientific component. For understanding programming, for teaching it and for applying it in the world, science is the wrong model. Mathematics has found its own foundations apart from science, and computer science must do the same.

Development of software using off-the-shelf components seems to offer a chance for improving product quality and developer productivity. This article reviews a foundational testing-based theory of component composition, describes tools that implement the theory, and presents experiments with functional and nonfunctional component/system properties that validate the theory and illuminate issues in component composition. The context for this work is an ideal form of Component-Based Software Development (CBSD) supported by tools. Component developers describe their components by measuring approximations to functional and nonfunctional behavior on a finite collection of subdomains. Systems designers describe an application-system structure by the component connections that form it. From measured component descriptions and a system structure, a CAD tool synthesizes the system properties, predicting how the system will behave. The system is not built, nor are any test executions performed. Neither the component sources nor executables are needed by systems designers. From CAD calculations a designer can learn (approximately) anything that could be learned by testing an actual system implementation. The CAD tool is often more efficient than it would be to assemble and execute an actual system. Using tools that support an ideal separation between component- and system development, experiments were conducted to investigate two related questions: (1) To what extent can unit (that is, component) testing replace system testing? (2) What properties of software and subdomains influence the quality of subdomain testing?

Abstract Composition of software elements into assemblies (systems) is a fundamental aspect of software development. It is an important strength of formal mathematical specification that the descriptions of elements can be precisely composed into the descriptions of assemblies. Testing, on the other hand, is usually thought to be ‘non‐compositional.’ Testing provides information about any executable software element, but testing descriptions have not been combined to describe assemblies of elements. The underlying reason for the compositional deficiency of testing is that tests are samples. When two elements are composed, the input samples (test points) for the first lead to an output sample, but it does not match the input test points of the second, following element. The current interest in software components and component‐based software development (CBSD) provides an ideal context for investigating elements and assemblies. In CBSD, the elements (components) are analysed without knowledge of the system(s) to be later assembled. A fundamental testing theory of component composition must use measured component properties (test results) to predict system properties. This paper proposes a testing‐based theory of software component composition based on subdomains. It shows how to combine subdomain tests of components into testing predictions for arbitrarily complex assemblies formed by sequence, conditional, and iteration constructions. The basic construction of the theory applies to functional behaviour, but the theory can also predict the system's non‐functional properties from component subdomain tests. Compared with the alternative of actually building and testing a system, theoretical predictions are computationally more efficient. The theory can also be described as an exercise in modelling. Components are replaced by abstractions derived from testing them, and these models are manipulated to model system behaviour. This article replaces a previously published version. DOI: 10.1002/stvr.368 . Copyright © 2007 John Wiley & Sons, Ltd.

Abstract The Term Redundancy Method (TRM) is a novel approach for obtaining ultra‐reliable programs through specification‐based testing. Current specification‐based testing schemes need a prohibitively large number of test cases for estimating ultra‐reliability. They assume the availability of an accurate program‐usage distribution prior to testing, and they assume the availability of a test oracle. This paper shows how to obtain ultra‐reliable abstract data types specified with equational specifications, with a practical number of test cases, without an accurate usage distribution, and without the usual test oracle. The effectiveness of the TRM in failure detection and recovery is demonstrated on the aircraft collision avoidance system TCAS. Copyright © 2007 John Wiley & Sons, Ltd.

This paper extends basic software-testing theory to software components and adds explicit state to the theory. The resulting theory is simple enough to abstractly model the construction of systems from their parts ('units'). It provides an unconventional insight into the relationship between testing units and testing systems. Experiments exploring the theory support the following conclusions:

Logical formulas called invariants are a staple of formal methods for program analysis. Persistent-state variables appear in these formulas playing their proper intuitive role, which is somewhere between inputs and internal variables. In software testing theory, on the contrary, state is not usually accorded explicit treatment. Comparing the viewpoints of formal methods and testing theory suggests new roles that formal methods can play in testing. This examination is motivated by recent systems such as Daikon, which use both tests and invariants.

The ICSE Doctoral Symposium is a forum for Ph.D. students to publicly discuss their research goals, methods, and results prior to completion of their doctorates. The Symposium aims to provide useful guidance for completion of the dissertation research and initiation of a research career. The Symposium and ICSE also provide an opportunity for student participants to interact with established researchers and others in the wider software engineering community.

Most engineering artifacts behave in a continuous fashion, and this property is generally believed to underlie their dependability. In contrast, software systems do not have continuous behavior, which is taken to be an underlying cause of their undependability. The theory of software reliability has been questioned because technically the sampling on which it is based applies only to continuous functions.This paper examines the role of continuity in engineering, particularly in testing and certifying artifacts, then considers the analogous software situations and the ways in which software is intrinsically unlike other engineered objects. Several definitions of software 'continuity' are proposed and related to ideas in software testing. It is shown how 'continuity' can be established in practice, and the consequences for testing and analysis of knowing that a program is 'continuous.Underlying any use of software 'continuity' is the continuity of its specification in the usual mathematical sense. However, many software applications are intrinsically discontinuous and one reason why software is so valuable is its natural ability to handle these applications, where it makes no sense to seek software 'continuity' or to blame poor dependability on its absence.

It is widely felt that software quality in the form of reliability or "trustworthiness", can be demonstrated by the successful completion of testing that "covers" the software. However, this intuition has little experimental or theoretical support. The paper considers why the intuition is so powerful and yet misleading. Formal definitions of software "dependability" are suggested, along with new approaches for measuring this analogy of trustworthiness.>