As firms increasingly rely on information systems to perform critical functions, the consequences of software defects can be catastrophic. Although the software engineering literature suggests that software process improvement can help to reduce software defects, the actual evidence is equivocal. For example, improved development processes may only remove the “easier” syntactical defects, while the more critical defects remain. Rigorous empirical analyses of these relationships have been very difficult to conduct due to the difficulties in collecting the appropriate data on real systems from industrial organizations. This field study analyzes a detailed data set consisting of 7,545 software defects that were collected on software projects completed at a major software firm. Our analyses reveal that higher levels of software process improvement significantly reduce the likelihood of high severity defects. In addition, we find that higher levels of process improvement are even more beneficial in reducing severe defects when the system developed is large or complex, but are less beneficial in development when requirements are ambiguous, unclear, or incomplete. Our findings reveal the benefits and limitations of software process improvement for the removal of severe defects and suggest where investments in improving development processes may have their greatest effects.

Abstract This research analyzes longitudinal empirical data on commercial software applications to test and better understand how software evolves over time, and to measure the likely long‐term effects of a software process automation tool on software productivity and quality. The research consists of two parts. First, we use data from source control systems, defect tracking systems, and archived project documentation to test a series of hypotheses developed by Belady and Lehman about software evolution. We find empirical support for many of these hypotheses, but not all. We then further analyze the data using moderated regression analysis to discern how software process automation efforts at the research site influenced the software evolution lifecycles of the applications. Our results support the claim that automation has enabled the organization to accomplish more work activities with greater productivity, thereby significantly increasing the functionality of the applications portfolio. Despite the growth in software functionality, the analysis suggests that automation has helped to manage software complexity levels and to improve quality by reducing errors over time. Our models and their results demonstrate how longitudinal empirical software data can be used to reveal the often elusive long‐term benefits of investments in software process improvement, and to help managers make more informed resource‐allocation decisions. Copyright © 2007 John Wiley & Sons, Ltd.

This research examines the structural complexity of software and, specifically, the potential interaction of the two dominant dimensions of structural complexity, coupling and cohesion. Analysis based on an information processing view of developer cognition results in a theoretically driven model with cohesion as a moderator for a main effect of coupling on effort. An empirical test of the model was devised in a software maintenance context utilizing both procedural and object-oriented tasks, with professional software engineers as participants. The results support the model in that there was a significant interaction effect between coupling and cohesion on effort, even though there was no main effect for either coupling or cohesion. The implication of this result is that, when designing, implementing, and maintaining software to control complexity, both coupling and cohesion should be considered jointly, instead of independently. By providing guidance on structuring software for software professionals and researchers, these results enable software to continue as the solution of choice for a wider range of richer, more complex problems.

Preparations for Y2K reminded the software engineering community of the extent to which long-lived software systems are embedded in our daily environments. As systems are maintained and enhanced throughout their lifecycles they appear to follow generalized behaviors described by the laws of software evolution. Within this context, however, there is some question of how and why systems may evolve differently. The objective of this work is to answer the question: do systems follow a set of identifiable evolutionary patterns? In this paper we use software volatility to describe the lifecycle evolution of a portfolio of 23 software systems. We show by example that a vector of software volatility levels can represent lifecycle behavior of a software system. We further demonstrate that the portfolio's 23 software volatility vectors can be grouped into four distinguishable patterns. Thus, we show by example that there are different patterns of system lifecycle behavior, i.e. software evolution.

With the approach of the new millennium, a primary focus in software engineering involves issues relating to upgrading, migrating, and evolving existing software systems. In this environment, the role of careful empirical studies as the basis for improving software maintenance processes, methods, and tools is highlighted. One of the most important processes that merits empirical evaluation is software evolution. Software evolution refers to the dynamic behaviour of software systems as they are maintained and enhanced over their lifetimes. Software evolution is particularly important as systems in organizations become longer-lived. However, evolution is challenging to study due to the longitudinal nature of the phenomenon in addition to the usual difficulties in collecting empirical data. We describe a set of methods and techniques that we have developed and adapted to empirically study software evolution. Our longitudinal empirical study involves collecting, coding, and analyzing more than 25000 change events to 23 commercial software systems over a 20-year period. Using data from two of the systems, we illustrate the efficacy of flexible phase mapping and gamma sequence analytic methods, originally developed in social psychology to examine group problem solving processes. We have adapted these techniques in the context of our study to identify and understand the phases through which a software system travels as it evolves over time. We contrast this approach with time series analysis. Our work demonstrates the advantages of applying methods and techniques from other domains to software engineering and illustrates how, despite difficulties, software evolution can be empirically studied.

Software maintenance is a task that is difficult to manage effectively. In part, this is because software managers have very little knowledge about the types of maintenance work that are likely to occur. If managers could forecast changes to software systems, they could more effectively plan, allocate workforce and manage change requests. But, the ability to forecast software modifications depends on whether there are predictable patterns in maintenance work. We posit that there are patterns in maintenance work and that certain characteristics of software modules are associated with these patterns. We examine modification profiles for 621 software modules in five different business systems of a commercial merchandiser. We find that only a small number of modules in these systems is likely to be modified frequently, and that certain maintenance patterns emerge. Modules frequently enhanced are in systems whose functionality is considered strategic. Modules frequently repaired have high software complexity, are large in size, and are relatively older. However, modules that have been code generated are less likely to be repaired. Older and larger modules are restructured and upgraded more frequently. Our results suggest that these characteristics of software modules are associated with predictable maintenance profiles. Such profile information can be used by software managers to predict and plan for maintenance more effectively. In addition, our results suggest the use of code generators as a means of reducing repair maintenance. © 1997 John Wiley & Sons, Ltd.

Many maintenance problems derive from inadequate software development practices. Poor design choices result in complex software that may be costly to support and difficult to change. A general framework based upon software complexity is proposed that could be used to assess the potential maintenance impact of development tools and techniques. To test the framework, a study was conducted in two commercial organizations. The results indicate that reduced maintenance effort is associated with development practices, including structured programming, report generators and packaged software. A surprising finding is that use of a software code generator is associated with increased maintenance effort.