Despite the increasing awareness of the importance of managing technical debt in software product development, systematic processes for implementing technical debt management in software production have not been readily available. In this paper we report on the development and field tests of a normative process framework that systematically incorporates steps for managing technical debt in commercial software production. The framework integrates processes required for technical debt management with existing software quality management processes prescribed by the project management body of knowledge (PMBOK), and it contributes to the further development of software-specific extensions to the PMBOK. We partnered with three commercial software product development organizations to implement the framework in real-world software production settings. All three organizations, irrespective of their varying software process maturity levels, were able to adopt the proposed framework and integrate the prescribed technical debt management processes with their existing software quality management processes. Our longitudinal observations and case-study interviews indicate that the organizations were able to accrue economic benefits from the adoption and use of the integrated framework.

Technical debt, defined as the maintenance obligations arising from shortcuts taken during the design, development, and deployment of software systems, has been shown to significantly impact the reliability and long-term evolution of software systems [1], [2]. Although academic research has moved beyond using technical debt only as a metaphor, and has begun compiling strong empirical evidence on the economic implications of technical debt, industry practitioners continue to find managing technical debt a challenging balancing act [3]. Despite the increasing awareness of the importance of managing technical debt in software product development, systematic processes for implementing technical debt management in software production have not been readily available.

We develop an evolutionary model and theory of software technical debt accumulation to facilitate a rigorous and balanced analysis of its benefits and costs in the context of a large commercial enterprise software package. Our theory focuses on the optimization problem involved in managing technical debt, and illustrates the different tradeoff patterns between software quality and customer satisfaction under early and late adopter scenarios at different lifecycle stages of the software package. We empirically verify our theory utilizing a ten year longitudinal data set drawn from 69 customer installations of the software package. We then utilize the empirical results to develop actionable policies for managing technical debt in enterprise software product adoption.

As firms increasingly rely on information systems to perform critical functions, the consequences of software defects can be catastrophic. Although the software engineering literature suggests that software process improvement can help to reduce software defects, the actual evidence is equivocal. For example, improved development processes may only remove the “easier” syntactical defects, while the more critical defects remain. Rigorous empirical analyses of these relationships have been very difficult to conduct due to the difficulties in collecting the appropriate data on real systems from industrial organizations. This field study analyzes a detailed data set consisting of 7,545 software defects that were collected on software projects completed at a major software firm. Our analyses reveal that higher levels of software process improvement significantly reduce the likelihood of high severity defects. In addition, we find that higher levels of process improvement are even more beneficial in reducing severe defects when the system developed is large or complex, but are less beneficial in development when requirements are ambiguous, unclear, or incomplete. Our findings reveal the benefits and limitations of software process improvement for the removal of severe defects and suggest where investments in improving development processes may have their greatest effects.

This study develops and empirically tests the idea that the impact of structural complexity on perfective maintenance of object-oriented software is significantly determined by the team strategy of programmers (independent or collaborative). We analyzed two key dimensions of software structure, coupling and cohesion, with respect to the maintenance effort and the perceived ease-of-maintenance by pairs of programmers. Hypotheses based on the distributed cognition and task interdependence theoretical frameworks were tested using data collected from a controlled lab experiment employing professional programmers. The results show a significant interaction effect between coupling, cohesion, and programmer team strategy on both maintenance effort and perceived ease-of-maintenance. Highly cohesive and low-coupled programs required lower maintenance effort and were perceived to be easier to maintain than the low-cohesive programs and high-coupled programs. Further, our results would predict that managers who strategically allocate maintenance tasks to either independent or collaborative programming teams depending on the structural complexity of software could lower their team's maintenance effort by as much as 70 percent over managers who use simple uniform resource allocation policies. These results highlight the importance of achieving congruence between team strategies employed by collaborating programmers and the structural complexity of software.

This research investigates the effect of review rate on defect removal effectiveness and the quality of software products, while controlling for a number of potential confounding factors. Two data sets of 371 and 246 programs, respectively, from a personal software process (PSP) approach were analyzed using both regression and mixed models. Review activities in the PSP process are those steps performed by the developer in a traditional inspection process. The results show that the PSP review rate is a significant factor affecting defect removal effectiveness, even after accounting for developer ability and other significant process variables. The recommended review rate of 200 LOC/hour or less was found to be an effective rate for individual reviews, identifying nearly two-thirds of the defects in design reviews and more than half of the defects in code reviews.

Abstract This research analyzes longitudinal empirical data on commercial software applications to test and better understand how software evolves over time, and to measure the likely long‐term effects of a software process automation tool on software productivity and quality. The research consists of two parts. First, we use data from source control systems, defect tracking systems, and archived project documentation to test a series of hypotheses developed by Belady and Lehman about software evolution. We find empirical support for many of these hypotheses, but not all. We then further analyze the data using moderated regression analysis to discern how software process automation efforts at the research site influenced the software evolution lifecycles of the applications. Our results support the claim that automation has enabled the organization to accomplish more work activities with greater productivity, thereby significantly increasing the functionality of the applications portfolio. Despite the growth in software functionality, the analysis suggests that automation has helped to manage software complexity levels and to improve quality by reducing errors over time. Our models and their results demonstrate how longitudinal empirical software data can be used to reveal the often elusive long‐term benefits of investments in software process improvement, and to help managers make more informed resource‐allocation decisions. Copyright © 2007 John Wiley & Sons, Ltd.

This research examines the structural complexity of software and, specifically, the potential interaction of the two dominant dimensions of structural complexity, coupling and cohesion. Analysis based on an information processing view of developer cognition results in a theoretically driven model with cohesion as a moderator for a main effect of coupling on effort. An empirical test of the model was devised in a software maintenance context utilizing both procedural and object-oriented tasks, with professional software engineers as participants. The results support the model in that there was a significant interaction effect between coupling and cohesion on effort, even though there was no main effect for either coupling or cohesion. The implication of this result is that, when designing, implementing, and maintaining software to control complexity, both coupling and cohesion should be considered jointly, instead of independently. By providing guidance on structuring software for software professionals and researchers, these results enable software to continue as the solution of choice for a wider range of richer, more complex problems.

Preparations for Y2K reminded the software engineering community of the extent to which long-lived software systems are embedded in our daily environments. As systems are maintained and enhanced throughout their lifecycles they appear to follow generalized behaviors described by the laws of software evolution. Within this context, however, there is some question of how and why systems may evolve differently. The objective of this work is to answer the question: do systems follow a set of identifiable evolutionary patterns? In this paper we use software volatility to describe the lifecycle evolution of a portfolio of 23 software systems. We show by example that a vector of software volatility levels can represent lifecycle behavior of a software system. We further demonstrate that the portfolio's 23 software volatility vectors can be grouped into four distinguishable patterns. Thus, we show by example that there are different patterns of system lifecycle behavior, i.e. software evolution.

With the approach of the new millennium, a primary focus in software engineering involves issues relating to upgrading, migrating, and evolving existing software systems. In this environment, the role of careful empirical studies as the basis for improving software maintenance processes, methods, and tools is highlighted. One of the most important processes that merits empirical evaluation is software evolution. Software evolution refers to the dynamic behaviour of software systems as they are maintained and enhanced over their lifetimes. Software evolution is particularly important as systems in organizations become longer-lived. However, evolution is challenging to study due to the longitudinal nature of the phenomenon in addition to the usual difficulties in collecting empirical data. We describe a set of methods and techniques that we have developed and adapted to empirically study software evolution. Our longitudinal empirical study involves collecting, coding, and analyzing more than 25000 change events to 23 commercial software systems over a 20-year period. Using data from two of the systems, we illustrate the efficacy of flexible phase mapping and gamma sequence analytic methods, originally developed in social psychology to examine group problem solving processes. We have adapted these techniques in the context of our study to identify and understand the phases through which a software system travels as it evolves over time. We contrast this approach with time series analysis. Our work demonstrates the advantages of applying methods and techniques from other domains to software engineering and illustrates how, despite difficulties, software evolution can be empirically studied.

With the increasing use of object-oriented methods in new software development, there is a growing need to both document and improve current practice in object-oriented design and development. In response to this need, a number of researchers have developed various metrics for object-oriented systems as proposed aids to the management of these systems. In this research, an analysis of a set of metrics proposed by Chidamber and Kemerer (1994) is performed in order to assess their usefulness for practising managers. First, an informal introduction to the metrics is provided by way of an extended example of their managerial use. Second, exploratory analyses of empirical data relating the metrics to productivity, rework effort and design effort on three commercial object-oriented systems are provided. The empirical results suggest that the metrics provide significant explanatory power for variations in these economic variables, over and above that provided by traditional measures, such as size in lines of code, and after controlling for the effects of individual developers.

Software maintenance is a task that is difficult to manage effectively. In part, this is because software managers have very little knowledge about the types of maintenance work that are likely to occur. If managers could forecast changes to software systems, they could more effectively plan, allocate workforce and manage change requests. But, the ability to forecast software modifications depends on whether there are predictable patterns in maintenance work. We posit that there are patterns in maintenance work and that certain characteristics of software modules are associated with these patterns. We examine modification profiles for 621 software modules in five different business systems of a commercial merchandiser. We find that only a small number of modules in these systems is likely to be modified frequently, and that certain maintenance patterns emerge. Modules frequently enhanced are in systems whose functionality is considered strategic. Modules frequently repaired have high software complexity, are large in size, and are relatively older. However, modules that have been code generated are less likely to be repaired. Older and larger modules are restructured and upgraded more frequently. Our results suggest that these characteristics of software modules are associated with predictable maintenance profiles. Such profile information can be used by software managers to predict and plan for maintenance more effectively. In addition, our results suggest the use of code generators as a means of reducing repair maintenance. © 1997 John Wiley & Sons, Ltd.

Given the central role that software development plays in the delivery and application of information technology, managers are increasingly focusing on process improvement in the software development area. This demand has spurred the provision of a number of new and/or improved approaches to software development, with perhaps the most prominent being object-orientation (OO). In addition, the focus on process improvement has increased the demand for software measures, or metrics with which to manage the process. The need for such metrics is particularly acute when an organization is adopting a new technology for which established practices have yet to be developed. This research addresses these needs through the development and implementation of a new suite of metrics for OO design. Metrics developed in previous research, while contributing to the field's understanding of software development processes, have generally been subject to serious criticisms, including the lack of a theoretical base. Following Wand and Weber (1989), the theoretical base chosen for the metrics was the ontology of Bunge (1977). Six design metrics are developed, and then analytically evaluated against Weyuker's (1988) proposed set of measurement principles. An automated data collection tool was then developed and implemented to collect an empirical sample of these metrics at two field sites in order to demonstrate their feasibility and suggest ways in which managers may use these metrics for process improvement.>

One measure of the size and complexity of information systems that is growing in acceptance and adoption is function points, a user-oriented, nonsource line of code metric of the systems development product. Previous research has documented the degree of reliability of function points as a metric. This research extends that work by (a) identifying the major sources of variation through a survey of current practice, and (b) estimating the magnitude of the effect of these sources of variation using detailed case study data from commercial systems. The results of this research show that a relatively small number of factors has the greatest potential for affecting reliability, and recommendations are made for using these results to improve the reliability of function point counting in organizations.>

"June 1991."

A study of the relationship between the cyclomatic complexity metric (T. McCabe, 1976) and software maintenance productivity, given that a metric that measures complexity should prove to be a useful predictor of maintenance costs, is reported. The cyclomatic complexity metric is a measure of the maximum number of linearly independent circuits in a program control graph. The current research validates previously raised concerns about the metric on a new data set. However, a simple transformation of the metric is investigated whereby the cyclomatic complexity is divided by the size of the system in source statements. thereby determining a complexity density ratio. This complexity density ratio is demonstrated to be a useful predictor of software maintenance productivity on a small pilot sample of maintenance projects.>

In this research we reconcile two opposing views regarding the presence of economies or diseconomies of scale in new software development Our general approach hypothesizes a production function model of software development that allows for both increasing and decreasing returns to scale, and argues that local scale economies or diseconomies depend upon the size of projects. Using eight different data sets, including several reported in previous research on the subject. we provide empirical evidence in support of our hypothesis. Through use of the nonparametric DBA technique we also show how to identify the most productive scale size that may vary across organizations. These results are extended to include the effects of nonparametric scale-related factors, such as project duration and the number of new staff on the project team.

Reports on efforts to develop and estimate a preliminary model of the software production process using pilot data from 65 software maintenance projects recently completed by a large regional bank's data processing department. The goals are to measure factors that affect software maintenance productivity, to integrate the quality and productivity dimensions of software measurements, and to examine the productivity of entire projects rather than only the programming phase, which typically accounts for less than half the effort on a software project. Variables relating to the quality of labor employed on the projects are included. To investigate the set of potential productivity factors, the technique of data envelopment analysis (DEA) is used to estimate the relationship between the inputs and products of software maintenance. The general approach to this research is to model software development as a microeconomic production process utilizing inputs and producing products.>