The objective of the work described is to accurately predict, as early as possible in the software lifecycle, how reliably a new software release will behave in the field. The initiative is based on a set of innovative mathematical models that have consistently shown a high correlation between key in-process metrics and our primary customer experience metric, SWDPMH (Software Defects per Million Hours [usage] per Month). We have focused on the three primary dimensions of testing – incoming, fixed, and backlog bugs. All of the key predictive metrics described here are empirically-derived, and in specific quantitative terms have not previously been documented in the software engineering/quality literature. A key part of this work is the empirical determination of the precision of the measurements of the primary predictive variables, and the determination of the prediction (outcome) error. These error values enable teams to accurately gauge bug finding and fixing progress, week by week, during the primary test period.

Cisco Systems, Inc., conducts a customer satisfaction survey (CSAT) each year to gauge customer sentiment regarding Cisco products, technical support, partner- and Cisco-provided technical services, order fulfillment, and a number of other aspects of the companys business. The results of the analysis of this data are used for several purposes, including ascertaining the viability of new products, determining if customer support objectives are being met, setting engineering in-process and customer experience yearly metrics goals, and assessing, indirectly, the success of engineering initiatives. Analyzing this data, which includes 110,000 yearly sets of survey responses that address over 100 product and services categories, is in many respects complicated. For example, skip logic is an integral part of the survey mechanics, and forming aggregate views of customer sentiment is statistically challenging in this data environment. In this paper, we describe several of the various analysis approaches currently used, pointing out some situations where a high level of precision is not easily achieved, and some situations in which it is possible to easily end up with erroneous results. The analysis and statistical territory covered in this paper is in parts well-known and straightforward, but other parts, which we address, are susceptible to large inaccuracies and errors. We address several of these difficulties and develop reasonable solutions for two known issues, high missing value levels and high colinearity of independent variables.

Over the past eight years, Cisco Systems, Inc., has implemented software quality goals for most groups engaged in software development, including development for both customer and internal use. This corporate implementation has proven to be a long and difficult process for many reasons, including opposition from many groups, uncertainties as to how to proceed with key aspects of the goaling, and the many unanticipated modifications needed to adapt the program to a large and diverse development and test environment. This paper describes what has worked, what has not work so well, and what levels of improvement the Engineering organization has experienced in part as a result of these efforts. Key customer experience metrics have improved 30% to 70% over the past six years, partly as a result of metrics and process standardization, dashboarding, and goaling. As one would expect with such a large endeavor, some of the results shown are not statistically provable, but are nevertheless generally accepted within the corporation as valid. Other important results do have strong statistical substantiation, and we will also describe these. But whether or not the results are statistically provable, Cisco has in fact improved its software quality substantially over the past eight years, and the corporate goaling mechanism is generally recognized as a necessary (but of course not sufficient) part of this improvement effort.

Software becomes ever more feature-rich and thereby harder to distinguish based on its functionality. Instead, quality is starting to differentiate between similar software products. Specifying, constructing, and assuring quality has been under research for several decades and continues to be a long- term research area because of its many facets and its com- plexity. Current national and international initiatives show that there is an active research community in academia and industry. This workshop builds on the rich experiences of a series of previous workshops and aims to bring this community together to discuss current issues and future developments.

This is the introduction of the 3rd workshop on Software Development Governance (SDG), which will take place as part of ICSE 2010. This year we have combined two successful workshops (SDG - software development governance and LMSA -- leadership and management in software architecture) since both workshops deal with decisions that are part of the development process e.g., business and organizational decisions that impact the technical decisions concerned with the product architecture and the product quality.

Cost, schedule and quality are highly correlated factors in software development. They basically form three sides of the same triangle. Beyond a certain point (the "Quality is Free" point), it is difficult to increase the quality without increasing either cost or schedule or both for the software under development. As products and applications mature, users expect higher quality products. They want IT organizations to be responsible and accountable for the quality claims made by the product marketing teams. In the last couple decades, much software engineering research has focussed on standards, methodologies and techniques for improving software quality, measuring software quality and software quality assurance. Most of this research is focused on the internal/development view of quality. More recent studies done in conjunction with the marketing groups have made attempts to understand the customer view of quality. All of these different ongoing activities to understand quality from the various perspectives have made the field even more enriching and exciting. The Fourth Workshop on Software Quality aims to bring together academic, industrial and commercial communities interested in software quality topics to discuss the different technologies being defined and used in the software quality area.

To develop software on time and within budget is not good enough if the software produced is full of defects. As the software market matures, users want to be assured of quality. They no longer accept claims made by the IT department at face value, but expect concrete demonstrations of quality. Software stakeholders are also demanding higher quality software than ever before. In recent years, much software engineering research has focussed on standards, methodologies and techniques for improving software quality, measuring software quality and software quality assurance. Most of this research is focused on an internal view of quality whereas few measures of the customer view of quality exist. The Third Workshop on Software Quality aims to bring together academic, industrial and commercial communities interested in software quality topics to discuss the different technologies being defined and used in the software quality area.

Due to the pervasive nature of software, software-engineering practitioners have continuously expressed their concerns over their inability to accurately predict the cost, schedule and quality of a software product under development. Thus, one of the most important objectives of the software engineering community has been to develop useful models that constructively explain the software development lifecycle and accurately predict the cost, schedule and quality of developing a software product. Most of the existing parametric models have been empirically calibrated to actual data from completed software projects. The most commonly used technique for empirical calibration has been the popular classical multiple regression approach. This approach imposes a few restrictions often violated by software engineering data and has resulted in the development of inaccurate empirical models that do not perform very well. The focus of this dissertation is to explain the drawbacks of the multiple regression approach for software engineering data and discuss the Bayesian approach which alleviates a few of the problems faced by the multiple regression approach.

Many parametric software estimation models have evolved in the last two decades (L.H. Putnam and W. Myers, 1992; C. Jones, 1997; R.M. Park et al., 1992). Almost all of these parametric models have been empirically calibrated to actual data from completed software projects. The most commonly used technique for empirical calibration has been the popular classical multiple regression approach. As discussed in the paper, the multiple regression approach imposes a few assumptions frequently violated by software engineering datasets. The paper illustrates the problems faced by the multiple regression approach during the calibration of one of the popular software engineering cost models, COCOMO II. It describes the use of a pragmatic 10 percent weighted average approach that was used for the first publicly available calibrated version (S. Chulani et al., 1998). It then moves on to show how a more sophisticated Bayesian approach can be used to alleviate some of the problems faced by multiple regression. It compares and contrasts the two empirical approaches, and concludes that the Bayesian approach was better and more robust than the multiple regression approach.