We were delighted to have the paper “A Methodology for Collecting Valid Software Engineering Data”, by Victor Basili and David Weiss (1984) considered one of the most influential papers of IEEE Transaction of Software Engineering’s first decade. The paper discusses data or information on software quality and what were the most effective software development techniques. It was published in November, 1984. It was written at a time when software development was just becoming widespread practice, but it was not common to use data or information on software quality in order to learn what were the most effective software development techniques under different conditions. To try to alleviate this problem, we suggested that when people (or organizations) developed software, they should try to measure the effectiveness of their development techniques. In particular, since software is typically long-living, change data should be collected so that methods and processes could be managed.

Determining whether systems achieve desired emergent properties, such as safety or reliability, requires an analysis of the system as a whole, often in later development stages when changes are difficult and costly to implement. In this article we propose the Process Risk Indicator (PRI) methodology for analyzing and evaluating emergent properties early in the development cycle. A fundamental assumption of system engineering is that risk mitigation processes reduce system risks, yet these processes may also be a source of risk: (1) processes may not be appropriate for achieving the desired emergent property; or (2) processes may not be followed appropriately. PRI analyzes development process artifacts (e.g., designs pertaining to reliability or safety analysis reports) to quantify process risks that may lead to higher system risk. We applied PRI to the hazard analysis processes of a network-centric, Department of Defense system-of-systems and two NASA spaceflight projects to assess the risk of not achieving one such emergent property, software safety, during the early stages of the development lifecycle. The PRI methodology was used to create measurement baselines for process indicators of software safety risk, to identify risks in the hazard analysis process, and to provide feedback to projects for reducing these risks.

The concept of "software quality" is often hard to capture for an organization. Quality models aim at making the concept more operational by refining the "quality" of software development products and processes into sub-concepts down to the level of concrete metrics and indicators. In practice, it is difficult for an organization to come up with a reliable quality model because quality depends on numerous organizational context factors, and the model as well as the metrics and indicators need to be tailored to the specifics of the organization. This paper presents experiences in developing custom-tailored quality models for an organization, exemplified by Ecopetrol, a Colombian oil and gas company. The general approach taken is illustrated and excerpts from the initial model are presented.

In this case study, we examine software safety risk in three flight hardware systems in NASA's Constellation spaceflight program. We applied our Technical and Process Risk Measurement (TPRM) methodology to the Constellation hazard analysis process to quantify the technical and process risks involving software safety in the early design phase of these projects. We analyzed 154 hazard reports and collected metrics to measure the prevalence of software in hazards and the specificity of descriptions of software causes of hazardous conditions. We found that 49-70% of 154 hazardous conditions could be caused by software or software was involved in the prevention of the hazardous condition. We also found that 12-17% of the 2013 hazard causes involved software, and that 23-29% of all causes had a software control. The application of the TRPM methodology identified process risks in the application of the hazard analysis process itself that may lead to software safety risk.

The definitive version can be found at : http://onlinelibrary.wiley.com/ Copyright Wiley [Full text of this article is not available in the UHRA]

Art. 46

Business value analysis (BVA) quantifies the factors that provide value and cost to an organization. It aims at capturing value, controlling risks, and capitalizing on opportunities. GQM+Strategies is an approach designed to aid in the definition and alignment of business goals, strategies, and an integrated measurement program at all levels in the organization. In this paper we describe how to perform business value analysis (BVA) using the GQM+Strategies approach. The integration of these two approaches provides a coupling of cost-benefit and risk analysis (value goals) with operationally measurable business goals and supports the evaluation of business goal success and the effectiveness of the chosen strategies. An application of the combined approach is provided to illustrate the feasibility of the proposed method. It deals with the business goal of modernizing the product for the evolving market.

Adapting new software processes and practices in organizational and academic environments requires training the developers and validating the applicability of the newly introduced activities. Investigating process conformance during training and understanding if programmers are able and willing to follow the specific steps are crucial to evaluating whether the process improves various software product quality factors. In this paper we present a process model independent approach to detect process non-conformance. Our approach is based on non-intrusively collected data captured by a version control system and provides the project manager with timely updates. Further, we provide evidence of the applicability of our approach by investigating process conformance in a five day training class on eXtreme Programming (XP) practices at the Leibniz Universität Hannover. Our results show that the approach enabled researchers to formulate minimal intrusive methods to check for conformance and that for the majority of the investigated XP practices violations could be detected.

Code smells are design flaws in object-oriented designs that may lead to maintainability issues in the further evolution of the software system. This study focuses on the evolution of code smells within a system and their impact on the change behavior (change frequency and size). The study investigates two code smells, God Class and Shotgun Surgery, by analyzing the historical data over several years of development of two large scale open source systems. The detection of code smells in the evolution of those systems was performed by the application of an automated approach using detection strategies. The results show that we can identify different phases in the evolution of code smells during the system development and that code smell infected components exhibit a different change behavior. This information is useful for the identification of risk areas within a software system that need refactoring to assure a future positive evolution.

Over the last 25+ years, the software community has been searching for the best models for estimating variables of interest (e.g., cost, defects, and fault proneness). However, little research has been done to improve the reliability of the estimates. Over the last decades, scope error and error analysis have been substantially ignored by the community. This work attempts to fill this gap in the research and enhance a common understanding within the community. Results provided in this study can eventually be used to support human judgment-based techniques and be an addition to the portfolio. The novelty of this work is that, we provide a way of detecting and handling the scope error arising from estimation models. The answer whether or not scope error will occur is a pre-condition to safe use of an estimation model. We also provide a handy procedure for dealing with outliers as to whether or not to include them in the training set for building a new version of the estimation model. The majority of the work is empirically based, applying computational intelligence techniques to some COCOMO model variations with respect to a publicly available cost estimation data set in the PROMISE repository.

In the past decades the software engineering community has proposed a large collection of software development life cycles, models, and processes. The goal of a major set of these processes is to assure that the product is finished within time and budget, and that a predefined set of functional and nonfunctional requirements (e.g. quality goals) are satisfied at delivery time. Based upon the assumption that there is a real relationship between the process applied and the characteristics of the product developed from that process, we developed a tool supported approach that uses process nonconformance detection to identify potential risks in achieving the required process characteristics. In this paper we present the approach and a feasibility study that demonstrates its use on a large-scale software development project in the aerospace domain. We demonstrate that our approach, in addition to meeting the criteria above, can be applied to a real system of reasonable size; can represent a useful and adequate set of rules of relevance in such an environment; and can detect relevant examples of process nonconformance that provide useful insight to the project manager.

Classification makes a significant contribution to advancing knowledge in both science and engineering. It is a way of investigating the relationships between the objects to be classified and identifies gaps in knowledge. Classification in engineering also has a practical application; it supports object selection. They can help mature software engineering knowledge, as classifications constitute an organized structure of knowledge items. Till date, there have been few attempts at classifying in software engineering. In this research, we examine how useful classifications in software engineering are for advancing knowledge by trying to classify testing techniques. The paper presents a preliminary classification of a set of unit testing techniques. To obtain this classification, we enacted a generic process for developing useful software engineering classifications. The proposed classification has been proven useful for maturing knowledge about testing techniques, and therefore, SE, as it helps to: 1) provide a systematic description of the techniques, 2) understand testing techniques by studying the relationships among techniques (measured in terms of differences and similarities), 3) identify potentially useful techniques that do not yet exist by analyzing gaps in the classification, and 4) support practitioners in testing technique selection by matching technique characteristics to project characteristics.

Experimental replications are very important to the advancement of empirical software engineering. Replications are one of the key mechanisms to confirm previous experimental findings. They are also used to transfer experimental knowledge, to train people, and to expand a base of experimental evidence. Unfortunately, experimental replications are difficult endeavors. It is not easy to transfer experimental know-how and experimental findings. Based on our experience, this paper discusses this problem and proposes a Framework for Improving the Replication of Experiments (FIRE). The FIRE addresses knowledge sharing issues both at the intra-group (internal replications) and inter-group (external replications) levels. It encourages coordination of replications in order to facilitate knowledge transfer for lower cost, higher quality replications and more generalizable results.

Often software systems are developed by organizations consisting of many teams of individuals working together. Brooks states in the Mythical Man Month book that product quality is strongly affected by organization structure. Unfortunately there has been little empirical evidence to date to substantiate this assertion. In this paper we present a metric scheme to quantify organizational complexity, in relation to the product development process to identify if the metrics impact failure-proneness. In our case study, the organizational metrics when applied to data from Windows Vista were statistically significant predictors of failure-proneness. The precision and recall measures for identifying failure-prone binaries, using the organizational metrics, was significantly higher than using traditional metrics like churn, complexity, coverage, dependencies, and pre-release bug measures that have been used to date to predict failure-proneness. Our results provide empirical evidence that the organizational metrics are related to, and are effective predictors of failure-proneness.

Software safety is one such product characteristic and this approach has been applied to identifying software safety insight areas and goals and developing early software safety measures, models and responses. Although the actual safety of a system cannot be verified during development, measures can reveal early insights into potential safety problems and risks. The approach and the example software measures presented are based on experience working with the safety engineering group on a large Department of Defense program.

GQM+Strategies is a measurement approach that builds on the well-tested GQM approach to planning and implementing software measurement. Although GQM has proven itself useful in a variety of industrial settings, one recognized weakness is the difficulty for GQM users to link software measurement goals to higher-level goals of the organization in which the software is being developed. This linkage is important, as it helps to justify software measurement efforts and allows measurement data to contribute to higher-level decisions. GQM+strategies provides mechanisms for explicitly linking software measurement goals, to higher-level goals for the software organization, and further to goals and strategies at the level of the entire business.

Providing robust decision support for software engineering (SE) requires the collection of data across multiple contexts so that one can begin to elicit the context variables that can influence the results of applying a technology. However, the task of comparing contexts is complex due to the large number of variables involved. This works extends a previous one in which we proposed a practical and rigorous process for identifying evidence and context information from SE papers. The current work proposes a specific template to collect context information from SE papers and an interactive approach to compare context information about these studies. It uses visualization and clustering algorithms to help the exploration of similarities and differences among empirical studies. This paper presents this approach and a feasibility study in which the approach is applied to cluster a set of papers that were independently grouped by experts.

The number of scientific publications is constantly increasing, and the results published on Empirical Software Engineering are growing even faster. Some software engineering publishers have begun to collaborate with research groups to make available repositories of software engineering empirical data. However, these initiatives are limited due to data ownership and privacy issues. As a result, many researchers in the area have adopted systematic reviews as a mean to extract empirical evidence from published material. Systematic reviews are labor intensive and costly. In this paper, we argue that the use of Information Extraction Tools can support systematic reviews and significantly speed up the creation of repositories of SE empirical evidence.

Summary form only given. Some software engineering instructors have incorporated projects that either simulate product development for a corporate customer or have students work directly with corporate sponsors to solve a problem or produce an artifact. The rationale for the activity is to provide "real world" experience, a foretaste of what a professional in the field will be asked to do. Other instructors may forgo the project as taking too much time from an already full curriculum or due to the difficulty of making and maintaining corporate connections. They rely on the lecture format to convey the material. Are there other reasons for including "project-type" activities in software engineering instruction over and above the practical experience they provide? Are there reasons to try to incorporate strategies other than lecture into courses? This paper reviews those areas of educational theory that speak to practical strategies to enhance learning and remembering. Constructivism as a way to view the learning process will be highlighted. Vygotsky's social interaction theory, and scaffolding will be tied to group endeavors. Bloom's hierarchy of the cognitive domain will be viewed in terms of writing course learning outcomes. Also discussed is the importance of the congruence of learning outcomes and modes of assessment to insure learning has occurred and to provide assurances to the Higher Education community that we are doing what we claim to do

Software engineering researchers are increasingly relying on the empirical approach to advance the state of the art. The level of empirical rigor and evidence required to guide software engineering research, however, can vary drastically depending on many factors. In this session we identify some of these factors through a discussion of the state of the art in performing empirical studies in software engineering, and we show how we can utilize the notion of empirical maturity to set and adjust the empirical expectations for software engineering research efforts.Regarding the state of the art in performing empirical studies, we will offer perspectives on two classes of study: those concerned with humans utilizing a technology, e.g., a person applying a methodology, a technique, or a tool, where human skills and the ability to interact with the technology are some of the primes issues, and those concerned with the application of the technology to an artifact, e.g., a technique or tool applied to a design or a program. In the first case, the emphasis is typically on issues like feasibility, usefulness, and then on effectiveness. The technology tends to be less well specified and based more on the experience and skills of the technology applier. In the second case, the emphasis is typically on the efficiency and effectiveness of the technology. The technology tends to be well defined and the assumption is that the individual skill and experience plays a less important role. We will discuss the set of factors that influence the design, implementation, and validity of these studies.Regarding empirical maturity and its implications on the SE community's expectations, we will provide examples of the large spectrum of studies with different maturity levels that can be performed to successfully support software engineering research. We will then identify and analyze the following aspects that are likely to impact a study's maturity level: technology (well-specified vs. under development), goals of the study (effectiveness vs. feasibility), type of design and analysis (controlled experiment vs. case study, quantitative vs. qualitative), control and specification of threats to validity (internal vs. external threats), dependence on context (in vivo vs. in vitro), relationship to previous empirical work (replicated on-site, replicated off-site, non-replicated, non-replicable), and purposes of the study (exploratory vs. confirmatory). We will lead a discussion on these key aspects that must be considered to assess the empirical maturity of a piece of work in the context of its research area and the empirical maturity of that area.

Although most scientific and engineering disciplines view empiricism as a basic aspect of their discipline, that view has not been the tradition in software engineering. There should be a symbiotic relationship between the development of theories and empirical studies that test and evolve that theory. This talk discusses of the role of empirical study in the understanding and improvement of the software product and process. It offers a personal, historical perspective of the use of empiricism through a series of example applications that demonstrate the various roles that empiricism can play. The examples are taken from the author's own experience and include the use of empirical study to improve an organization's product quality and productivity in the Software Engineering Laboratory at NASA/Goddard and to evaluate and evolve the effectiveness of various software technologies for use in improving the dependability of software. It also suggests research directions for empirical study in software engineering.

While there are strong beliefs within the community about whether one particular parallel programming model is easier to use than another, there has been little research to analyze these claims empirically. Currently, the most popular paradigm is message-passing, as implemented by the MPI library [1]. However, MPI is considered to be difficult for developing programs, because it forces the programmer to work at a very low level of abstraction. One alternative parallel programming model is the PRAM model, which supports fine-grained parallelism and has a substantial history of algorithmic theory [2]. It is not possible to program current parallel machines using the PRAM model because modern architectures are not designed to support such a model efficiently. However, current trends towards multicore chips suggest that large-scale, fine-grained uniform-memory access parallel machines may soon be feasible. XMT-C is an extension of the C language that supports parallel directives to provide a PRAM-like model to the programmer. A prototype compiler exists that generates code which runs on a simulator for an XMT architecture [3].To better understand how much benefit a PRAM-like model could provide over a message-passing model, we conducted a feasibility study in an academic setting to compare the effort required to solve a particular problem. The questions under study were: can we measure the effort in developing a program using these two programming models and can we differentiate the amount of effort for each model?.The subjects participating in the study were divided up into two groups. One group solved a problem using the MPI library in either C,C++, or Fortran, and the other group solved the problem using XMT-C. The task was to write a function to multiply a sparse matrix with a dense vector. To obtain subjects, we leveraged existing graduate-level parallel programming courses at two different universities: University of California, Santa Barbara (UCSB), and University of Maryland (UMD). At UCSB, the students solved the problem in MPI, and at UMD, the students solved the same problem in XMT-C. The focus of the UCSB class was on developing parallel programs to run on the current generation of architectures, and the students were taught MPI, as well as other models. The focus of the UMD class was parallel algorithms in the PRAM model, and the students were taught XMT-C. The students were assigned to treatment groups by class. The study was integrated into each class, as the problem was a required assignment in each class.Subjects kept track of their effort with a self-reported effort log. We also collected automatic effort data by instrumenting the compilers, which recorded data at each compile. We computed three effort measures: self-reported, instrumented, and combined. Self-reported effort measures are based entirely on effort logs, instrumented effort measures are based entirely on timestamps from the instrumented compilers, and combined effort measures are based on compiler timestamps when the subject is working on the instrumented machine, and self-reported effort when the subject is working off the instrumented machine.The results of this preliminary study answer both of our questions in the positive. In this case, on average, students required less effort to solve the problem using XMT-C compared to MPI. The reduction in mean effort was approximately 50% for all three measures, which was statistically significant at the level of p < .05 using a t-test.This study demonstrates that the effect of programming model on effort can be directly measured through empirical studies with human subjects. While no single study can conclusively demonstrate the advantage of one programming model over another, a series of studies examining different models and different problems can provide insights into the relative strengths of parallel programming models in different contexts. The study described above is one of a series of such studies that we are currently conducting.

In developing High-Performance Computing (HPC) software, time to solution is an important metric. This metric is comprised of two main components: the human effort required developing the software, plus the amount of machine time required to execute it. To date, little empirical work has been done to study the first component: the human effort required and the effects of approaches and practices that may be used to reduce it. In this paper, we describe a series of studies that address this problem. We instrumented the development process used in multiple HPC classroom environments. We analyzed data within and across such studies, varying factors such as the parallel programming model used and the application being developed, to understand their impact on the development process.

Dealing with dependability requirements is a complex task for stakeholders and analysts as many different aspects of a system must be taken into account at the same time: services characteristics and quality properties, failure modes and tolerable failure rates, reactions and recovery time in case of failure, and so on. Visualization helps cope with this complexity. In this paper, we build upon a practical framework for eliciting and modeling dependability requirements to show how graphical data representation can facilitate requirements analysis during the requirements elicitation and definition process. An Air Traffic Control System, adopted as a Testbed within the NASA High Dependability Computing Project, is used as a case study.

Measuring effort accurately and consistently across subjects in a programming experiment can be a surprisingly difficult task. In particular, measures based on self-reported data may differ significantly from measures based on data which is recorded automatically from a subject's computing environment. Since self-reports can be unreliable, and not all activities can be captured automatically, a complete measure of programming effort should incorporate both classes of data. In this paper, we show how self-reported and automatic effort can be combined to perform validation and to measure total programming effort.

Experience management refers to the capture, structuring, analysis, synthesis, and reuse of an organization's experience in the form of documents, plans, templates, processes, data, etc. The problem of managing experience effectively is not unique to software development, but the field of software engineering has had a high-level approach to this problem for some time. The Experience Factory is an organizational infrastructure whose goal is to produce, store, and reuse experiences gained in a software development organization. This paper describes The Q-Labs Experience Management System (Q-Labs EMS), which is based on the Experience Factory concept and was developed for use in a multinational software engineering consultancy. A critical aspect of the Q-Labs EMS project is its emphasis on empirical evaluation as a major driver of its development and evolution. The initial prototype requirements were grounded in the organizational needs and vision of Q-Labs, as were the goals and evaluation criteria later used to evaluate the prototype. However, the Q-Labs EMS architecture, data model, and user interface were designed to evolve, based on evolving user needs. This paper describes this approach, including the evaluation that was conducted of the initial prototype and its implications for the further development of systems to support software experience management.

Selecting technologies for developing software is a crucial activity in software projects. Defect reduction is an example of an area in which software developers have to decide what technologies to use. CeBASE is a NSF funded project that has the role of improving software development by providing decision support on the selection of techniques and tools. The decision support is based on empirical data organized in experience bases and refined into high-level models. Empirical data is collected through various activities, for example through eWorkshops in which experts discuss important issues, and formalized using the lightweight knowledge dust to knowledge pearl process.

For 25 years the NASA/GSFC Software Engineering Laboratory (SEL) has been a major resource in software process improvement activities. But due to a changing climate at NASA, agency reorganization, and budget cuts, the SEL has lost much of its impact. In this paper we describe the history of the SEL and give some lessons learned on what we did right, what we did wrong, and what others can learn from our experiences. We briefly describe the research that was conducted by the SEL, describe how we evolved our understanding of software process improvement, and provide a set of lessons learned and hypotheses that should enable future groups to learn from and improve on our quarter century of experiences.

One of the main problems in software testing is the development of a suitable set of test cases so that the effectiveness of the test is maximised with a minimum number of test cases. A lot of testing techniques are now available for developing test cases. However, some of them are misused, others are never used and only a few are applied again and again. When developers have to decide what testing techniques(s) they should use in a project, they have little (if any) experiential information about the available testing techniques, their usefulness and, in general, how suited they are to the project. This paper presents the results of developing a characterization scheme for test technique selection. When instantiated for different techniques, the scheme should provide developers with enough information for choosing the best suited to their project. Thus, their decisions would be based on sound knowledge of the techniques, instead of perceptions, suppositions and assumptions.

We propose an approach (GOM/MEDEA) for defining measures of product attributes in software engineering. The approach is driven by the experimental goals of measurement, expressed via the GQM paradigm, and a set of empirical hypotheses. To make the empirical hypotheses quantitatively verifiable, GQM/MEDEA supports the definition of theoretically valid measures for the attributes of interest based on their expected mathematical properties. The empirical hypotheses are subject to experimental verification. This approach integrates several research contributions from the literature into a consistent, practical, and rigorous approach.

No abstract available.

The work described in this paper is an investigation of COTS-based software development within a particular NASA environment, with an emphasis on the processes used. Fifteen projects using a COTS-based approach were studied and their actual process was documented. This process is evaluated to identify essential differences in comparison to traditional software development. The main differences, and the activities for which projects require more guidance, are requirements definition and COTS selection, high level design, integration and testing.

Software organizations are in need of methods to understand, structure, and improve the data their are collecting. We have developed an approach for use when a large number of diverse metrics are already being collected by a software organization (M.G. Mendonca et al., 1998; M.G. Mendonca, 1997). The approach combines two methods. One looks at an organization's measurement framework in a top-down goal-oriented fashion and the other looks at it in a bottom-up data-driven fashion. The top-down method is based on a measurement paradigm called Goal-Question-Metric (GQM). The bottom-up method is based on a data mining technique called Attribute Focusing (AF). A case study was executed to validate this approach and to assess its usefulness in an industrial environment. The top-down and bottom-up methods were applied in the customer satisfaction measurement framework at the IBM Toronto Laboratory. The top-down method was applied to improve the customer satisfaction (CUSTSAT) measurement from the point of view of three data user groups. It identified several new metrics for the interviewed groups, and also contributed to better understanding of the data user needs. The bottom-up method was used to gain new insights into the existing CUSTSAT data. Unexpected associations between key variables prompted new business insights, and revealed problems with the process used to collect and analyze the CUSTSAT data. The paper uses the case study and its results to qualitatively compare our approach against current ad hoc practices used to improve existing measurement frameworks.

The empirical study described in the paper addresses software reading for construction: how application developers obtain an understanding of a software artifact for use in new system development. The study focuses on the processes that developers would engage in when learning and using object oriented frameworks. We analyzed 15 student software development projects using both qualitative and quantitative methods to gain insight into what processes occurred during framework usage. The contribution of the study is not to test predefined hypotheses but to generate well-supported hypotheses for further investigation. The main hypotheses produced are that example based techniques are well suited to use by beginning learners, while hierarchy based techniques are not, because of a larger learning curve. Other more specific hypotheses are proposed and discussed.

No abstract available.

The use of commercial-off-the-shelf (COTS) products creates a software integration problem, whether a single COTS software component is being integrated into a software system, or the whole system is being built primarily from COTS products. This integration may require considerable effort and affect system quality. A good estimate of integration cost can help in the decision of whether or not to use a COTS solution, the selection of the best COTS products, and determine the amount and type of glueware that needs to be built. In this paper, we introduce a set of variables that have the potential to estimate the integration cost. We present a classification scheme of software architectures with respect to the integration of COTS products. The scheme is based on inter-component interactions within software architectures. The classification scheme allows the comparison of integration costs of different COTS products relative to different software architectures.

Inspections can be used to identify defects in software artifacts. In this way, inspection methods help to improve software quality, especially when used early in software development. Inspections of software design may be especially crucial since design defects (problems of correctness and completeness with respect to the requirements, internal consistency, or other quality attributes) can directly affect the quality of, and effort required for, the implementation.

Experimentation in software engineering is necessary but difficult. One reason is that there are a large number of context variables and, so, creating a cohesive understanding of experimental results requires a mechanism for motivating studies and integrating results. It requires a community of researchers that can replicate studies, vary context variables, and build models that represent the common observations about the discipline. The paper discusses the experience of the authors, based upon a collection of experiments, in terms of a framework for organizing sets of related studies. With such a framework, experiments can be viewed as part of common families of studies, rather than being isolated events. Common families of studies can contribute to important and relevant hypotheses that may not be suggested by individual experiments. A framework also facilitates building knowledge in an incremental manner through the replication of experiments within families of studies. To support the framework, the paper discusses the experiences of the authors in carrying out empirical studies, with specific emphasis on persistent problems encountered in experimental design, threats to validity, criteria for evaluation, and execution of experiments in the domain of software engineering.

The availability of significant measures in the early phases of the software development life-cycle allows for better management of the later phases, and more effective quality assessment when quality can be more easily affected by preventive or corrective actions. We introduce and compare various high-level design measures for object-based software systems. The measures are derived based on an experimental goal, identifying fault-prone software parts, and several experimental hypotheses arising from the development of Ada systems for Flight Dynamics Software at the NASA Goddard Space Flight Center (NASA/GSFC). Specifically, we define a set of measures for cohesion and coupling, which satisfy a previously published set of mathematical properties that are necessary for any such measures to be valid. We then investigate the measures' relationship to fault-proneness on three large scale projects, to provide empirical support for their practical significance and usefulness.

The empirical study described in this paper focuses on the effectiveness of maintenance processes in an environment in which a repository of potential sources of reuse exists, e.g. a context in which applications are built using an object-oriented framework. Such a repository might contain current and previous releases of the system under maintenance, as well as other applications that are built on a similar structure or contain similar functionality. This paper presents an observational study of 15 student projects in framework-based environment. We used a mix of qualitative and quantitative methods to identify and evaluate the effectiveness of the maintenance processes.

In this paper, we propose a qualitative, inductive method for characterizing and evaluating software maintenance processes, thereby identifying their specific problems and needs. This method encompasses a set of procedures which attempt to determine causal links between maintenance problems and flaws in the maintenance organization and process. This allows for a set of concrete steps to be taken for maintenance quality and productivity improvement, based on a tangible understanding of the relevant maintenance issues in a particular maintenance environment. Moreover, this understanding provides a solid basis on which to define relevant software maintenance models and measures. A case study of the application of this method, called Q-MOPP, is presented to further illustrate its feasibility and benefits. © 1998 John Wiley & Sons, Ltd.

This paper describes an empirical study that addresses the issue of communication among members of a software development organization. In particular, data was collected concerning code inspections in one software development project. The question of interest is whether or not organizational structure (the network of relationships between developers) has an effect on the amount of effort expended on communication between developers. The independent variables in this study are various attributes of the organizational structure in which the inspection participants work. The dependent variables are measures of the communication effort expended in various parts of the code inspection process, focusing on the inspection meeting. Both quantitative and qualitative methods were used, including participant observation, structured interviews, generation of hypotheses from field notes, statistical tests of relationships, and interpretation of results with qualitative anecdotes. The study results show that past and present working relationships between inspection participants affect the amount of meeting time spent in different types of discussion, thus affecting the overall inspection meeting length. Reporting relationships and physical proximity also have an effect. The contribution of the study is a set of well-supported hypotheses for further investigation.

In this paper we characterize and model the cost of rework in a Component Factory (CF) organization.A CF is responsible for developing and packaging reusable software components.Data was collected on corrective maintenance activities for the Generalized Support Software reuse asset library located at the Flight Dynamics Division of NASA's GSFC.We then constructed a predictive model of the cost of rework using the C4.5 system for generating a logical classification model.The predictor variables for the model are measures of internal software product attributes.The model demonstrates good prediction accuracy, and can be used by managers to allocate resources for corrective maintenance activities.Furthermore, we used the model to generate proscriptive coding guidelines to improve programming practices so that the cost of rework can be reduced in the future.The general approach we have used is applicable to other environments.

No abstract available.

This paper describes an empirical study which addresses the issue of communication among members of a software development organization.In particular, data was collected concerning code inspections in one software development project.The question of interest is whether or not organizational structure (the network of relationships between developers) has an effect on the amount of effort expended on communication between developers.Both quantitative and qualitative methods were used, including participant observation, structured interviews, generation of hypotheses from field notes, some simple statistical tests of relationships, and interpretation of results with qualitative anecdotes.The study results show that past and present working relationships between inspection participants affect the amount of meeting time spent in different types of discussion, thus affecting the overall meeting length.Reporting relationships and physical proximity also have an effect, as well as the point in the project that an inspection occurs.All but the last of these factors are organizational structure relationships.The contribution of the study is a set of well-supported hypotheses for further investigation.

First Page of the Article""

Component reuse is widely considered vital for obtaining significant improvement in development productivity. However, as an organization adopts a reuse-oriented development process, the nature of the problems in development is likely to change. In this article, we use a measurement-based approach to better understand and evaluate an evolving reuse process. More specifically, we study the effects of reuse across seven projects in narrow domain from a single development organization. An analysis of the errors that occur in new and reused components across all phases of system development provides insight into the factors influencing the reuse process. We found significant differences between errors associated with new and various types of reused components in terms of the types of errors committed. In addition, we identified differences when errors are introduced and the effect that the errors have on the development process.

S.196-197 : Lit.

A view of software measurement that disagrees with the model presented by Kitchenham, Pfleeger, and Fenton (1995), is given. Whereas Kitchenham et al. argue that properties used to define measures should not constrain the scale type of measures, the authors contend that that is an inappropriate restriction. In addition, a misinterpretation of Weyuker's (1988) properties is noted.

The paper presents a knowledge-based analysis approach that generates first order predicate logic annotations of loops. A classification of loops according to their complexity levels is presented. Based on this taxonomy, variations on the basic analysis approach that best fit each of the different classes are described. In general, mechanical annotation of loops is performed by first decomposing them using data flow analysis. This decomposition encapsulates closely related statements in events, that can be analyzed individually. Specifications of the resulting loop events are then obtained by utilizing patterns, called plans, stored in a knowledge base. Finally, a consistent and rigorous functional abstraction of the whole loop is synthesized from the specifications of its individual events. To test the analysis techniques and to assess their effectiveness, a case study was performed on an existing program of reasonable size. Results concerning the analyzed loops and the plans designed for them are given.

This paper presents the results of a study in which we empirically investigated the suite of object-oriented (OO) design metrics introduced in (Chidamber and Kemerer, 1994). More specifically, our goal is to assess these metrics as predictors of fault-prone classes and, therefore, determine whether they can be used as early quality indicators. This study is complementary to the work described in (Li and Henry, 1993) where the same suite of metrics had been used to assess frequencies of maintenance changes to classes. To perform our validation accurately, we collected data on the development of eight medium-sized information management systems based on identical requirements. All eight projects were developed using a sequential life cycle model, a well-known OO analysis/design method and the C++ programming language. Based on empirical and quantitative analysis, the advantages and drawbacks of these OO metrics are discussed. Several of Chidamber and Kemerer's OO metrics appear to be useful to predict class fault-proneness during the early phases of the life-cycle. Also, on our data set, they are better predictors than "traditional" code metrics, which can only be collected at a later phase of the software development processes.

Little theory exists in the field of software system measurement. Concepts such as complexity, coupling, cohesion or even size are very often subject to interpretation and appear to have inconsistent definitions in the literature. As a consequence, there is little guidance provided to the analyst attempting to define proper measures for specific problems. Many controversies in the literature are simply misunderstandings and stem from the fact that some people talk about different measurement concepts under the same label (complexity is the most common case). There is a need to define unambiguously the most important measurement concepts used in the measurement of software products. One way of doing so is to define precisely what mathematical properties characterize these concepts, regardless of the specific software artifacts to which these concepts are applied. Such a mathematical framework could generate a consensus in the software engineering community and provide a means for better communication among researchers, better guidelines for analysts, and better evaluation methods for commercial static analyzers for practitioners. We propose a mathematical framework which is generic, because it is not specific to any particular software artifact, and rigorous, because it is based on precise mathematical concepts. We use this framework to propose definitions of several important measurement concepts (size, length, complexity, cohesion, coupling). It does not intend to be complete or fully objective; other frameworks could have been proposed and different choices could have been made. However, we believe that the formalisms and properties we introduce are convenient and intuitive. This framework contributes constructively to a firmer theoretical ground of software measurement.

One important component of a software process is the organizational context in which the process is enacted.This component is ofien missing or incomplete in current process modeling approaches.One technique for modeling this perspective is the Actor-Dependency (AD) Model.This paper reports on a case study which used this approach to analyze and assess a large software maintenance organization.Our goal was to identifi the approach's strengths and weaknesses while providing practical recommendations for improvement and research directions.The AD model was found to be very useful in capturing the important properties of the organizational context of the maintenance process, and aided in the understanding of the flaws found in this process.However, a number of opportunities for extending and improving the AD model were identified.Among others, there is a need to incorporate quantitative information to complement the qualitative model.

Software requirements specifications (SRS) are often validated manually. One such process is inspection, in which several reviewers independently analyze all or part of the specification and search for faults. These faults are then collected at a meeting of the reviewers and author(s). Usually, reviewers use Ad Hoc or Checklist methods to uncover faults. These methods force all reviewers to rely on nonsystematic techniques to search for a wide variety of faults. We hypothesize that a Scenario-based method, in which each reviewer uses different, systematic techniques to search for different, specific classes of faults, will have a significantly higher success rate. We evaluated this hypothesis using a 3/spl times/2/sup 4/ partial factorial, randomized experimental design. Forty eight graduate students in computer science participated in the experiment. They were assembled into sixteen, three-person teams. Each team inspected two SRS using some combination of Ad Hoc, Checklist or Scenario methods. For each inspection we performed four measurements: (1) individual fault detection rate, (2) team fault detection rate, (3) percentage of faults first identified at the collection meeting (meeting gain rate), and (4) percentage of faults first identified by an individual, but never reported at the collection meeting (meeting loss rate). The experimental results are that (1) the Scenario method had a higher fault detection rate than either Ad Hoc or Checklist methods, (2) Scenario reviewers were more effective at detecting the faults their scenarios are designed to uncover, and were no less effective at detecting other faults than both Ad Hoc or Checklist reviewers, (3) Checklist reviewers were no more effective than Ad Hoc reviewers, and (4) Collection meetings produced no net improvement in the fault detection rate-meeting gains were offset by meeting losses.>

In order to improve software maintenance processes, we need to be able to first characterize and assess them. This task needs to be performed in depth and with objectivity since the problems are complex. One approach is to set up a measurement program specifically aimed at maintenance. However, establishing a measurement program requires that one understands the issues and is able to characterize the maintenance environment and processes in order to collect suitable and cost-effective data. Also, enacting such a program and getting usable data sets takes time. A short term substitute is needed. We propose a characterization process aimed specifically at maintenance and based on a general qualitative analysis methodology. This process is rigorously defined in order to be repeatable and usable by people who are not acquainted with such analysis procedures. A basic feature of our approach is that maintenance changes are analyzed in order to understand the flaws in the change process. Guidelines are provided and a case study is shown that demonstrates the usefulness of the approach.>

An overview of a knowledge-based approach which helps in the mechanical documentation and understanding of computer programs is given. This approach performs mechanical annotation of loops by first decomposing them into fragments, called events. It then recognizes the high-level concepts, represented by the events, based on patterns, called plans, stored in a knowledge-base. The design and utilization of the plans are described in detail, and how to generalize their structure is discussed. The generalized tree structure can facilitate plan recognition and reduce the size of the knowledge-base. A case study on a real program of some practical importance, containing a set of 77 loops, has been performed. Results concerning the plans designed for this case study are given.>

Software architecture appears to be one of the main factors affecting software maintainability. Therefore, in order to be able to predict and assess maintainability early in the development process one needs to be able to measure the high-level design characteristics that affect the change process. To this end, a measurement approach based on precise assumptions derived from the change process is proposed. The change process is based on object-oriented design principles and is partially language independent. Metrics for cohesion, coupling, and visibility are defined in order to capture the difficulty of isolating, understanding, designing and validating changes.>

This paper reports on the development of the data binding tool and its use in Ada source code reusability and software system design assessment. The tool was built around the metric of data bindings. Data bindings fall in the category of data visibility metrics and are used to measure inter-component interactions. Software system components are defined in the context of the Ada language using a flexible scheme. They are used, along with cluster analysis, to present structural configurations of a software system. The clustering technique as well as the tool design and its problems are discussed. The analysis of dendrograms (trees of components produced by the tool) reveals several classes of systems dendrograms and provides a simple mechanism for Ada source code reusability. Finally, the implications of different design methodologies used to develop the test software are discussed and explanations for the several types of dendrogram formulations are given.

Applying equal testing and verification effort to all parts of a software system is not very efficient, especially when resources are tight. Therefore, one needs to low/high fault frequency components so that testing/verification effort can be concentrated where needed. Such a strategy is expected to detect more faults and thus improve the resulting reliability of the overall system. The authors present the optimized set reduction approach for constructing such models, which is intended to fulfill specific software engineering needs. The approach to classification is to measure the software system and build multivariate stochastic models for predicting high-risk system components. Experimental results obtained by classifying Ada components into two classes (is, or is not likely to generate faults during system and acceptance rest) are presented. The accuracy of the model and the insights it provides into the error-making process are evaluated.>

Article Free Access Share on The software engineering laboratory: an operational software experience factory Authors: Victor Basili View Profile , Gianluigi Caldiera View Profile , Frank McGarry View Profile , Rose Pajerski View Profile , Gerald Page View Profile , Sharon Waligora View Profile Authors Info & Claims ICSE '92: Proceedings of the 14th international conference on Software engineeringJune 1992 Pages 370–381https://doi.org/10.1145/143062.143154Online:01 June 1992Publication History 50citation892DownloadsMetricsTotal Citations50Total Downloads892Last 12 Months30Last 6 weeks5 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

The authors address the issue of assessing the difficulty of a change based on known or predictable data. They consider the present work as a first step towards the construction of customized economic models for maintainers. They propose a modeling approach, based on regular statistical techniques, that can be used in a variety of software maintenance environments. This approach can be easily automated, and is simple for people with limited statistical experience to use. Moreover, it deals effectively with the uncertainty usually associated with both model inputs and outputs. The modeling approach is validated on a data set provided by the NASA Goddard Space Flight Center, showing that it has been effective in classifying changes with respect to the effort involved in implementing them. Other advantages of the approach are discussed along with additional steps to improve the results.

Applying equal testing and verification effort to all parts of a software system is not very efficient, especially when resources are limited and scheduling is tight. Therefore, one needs to be able to differentiate low/high fault density components so that testing/verification effort can be concentrated where needed. Such a strategy is ejected to detect more faults and thus improve the resulting reliability of the overall system. The authors present an alternative approach for constructing such models that is intended to fulfil specific software engineering needs, (i.e. dealing with partial/incomplete information and creating models that are easy to interpret). The approach to classification is to: measure the software system to be considered; and to build multivariate stochastic models for prediction. The authors present experimental results obtained by classifying FORTRAN components into two fault density classes: low and high. They also evaluate the accuracy of the model and the insights it provides into the software process.>

Software reuse can be achieved through an organization that focuses on utilization of life cycle products from previous developments. The component factory is both an example of the more general concepts of experience and domain factory and an organizational unit worth being considered independently. The critical features of such an organization are flexibility and continuous improvement. In order to achieve these features we can represent the architecture of the factory at different levels of abstraction and define a reference architecture from which specific architectures can be derived by instantiation. A reference architecture is an implementation and organization independent representation of the component factory and its environment. The paper outlines this reference architecture, discusses the instantiation process, and presents some examples of specific architectures by comparing them in the framework of the reference model.

In order to plan, control, and evaluate the software development process, one needs to collect and analyze data in a meaningful way. Classical techniques for such analysis are not always well suited to software engineering data. A pattern recognition approach for analyzing software engineering data, called optimized set reduction (OSR), that addresses many of the problems associated with the usual approaches is described. Methods are discussed for using the technique for prediction, risk management, and quality evaluation. Experimental results are provided to demonstrate the effectiveness of the technique for the particular application of software cost estimation.>

A methodology and a knowledge representation and reasoning framework for top-down goal-oriented characterization, modeling, and execution of software engineering activities is presented. A prototype system (ES-TAME) which demonstrates the underlying knowledge representation and reasoning principles is described. ES-TAME provides an object-oriented metamodel concept that provides support for tailorable and reusable software engineering models (SEMs). It provides the basic mechanisms, functions, and attributes for all the other models. It is based on interobject relationships, dynamic viewpoints, and selective inheritance in addition to traditional object-oriented mechanisms. Descriptive SEMs include representations for basic software engineering activities. They are controlled and made operational by active GQM (goal-question-metric paradigm) models which are built by a systematic mechanism for defining and evaluating project and corporate goals and using measurement to provide feedback in real-time.>

The problems related to the analysis and reengineering of existing software in order to reuse it are discussed. The authors introduce a process model for component extraction and focus on the problem of analyzing and qualifying software components which are candidates for reuse. A prototype tool for supporting the extraction of reusable components is presented. One of the components of this tool aids in understanding programs and is based on the functional model of correctness. It can assist software engineers in the process of finding correct formal specifications for programs. A detailed description of this component and an example to demonstrate a possible operational scenario are given.>

Using measures of data interaction called data bindings, the authors quantify ratios of coupling and strength in software systems and use the ratios to identify error-prone system structures. A 148000 source line system from a prediction environment was selected for empirical analysis. Software error data were collected from high-level system design through system testing and from field operation of the system. The authors use a set of five tools to calculate the data bindings automatically and use a clustering technique to determine a hierarchical description of each of the system's 77 subsystems. A nonparametric analysis of variance model is used to characterize subsystems and individual routines that had either many or few errors or high or low error correction effort. The empirical results support the effectiveness of the data bindings clustering approach for localizing error-prone system structure.>

A new paradigm for software development that treats software development as an experimental activity is presented. Built-in mechanisms are provided for learning how to develop software better and reusing previous experience in the forms of knowledge, processes, and products. Models and measures are used to aid in the tasks of characterization, evaluation, and motivation. An organization scheme is proposed for separating the project-specific focus from the organization's learning and reuse focuses of software development. The implications of this approach for corporations, research, and education are discussed, and some research activities currently underway at the University of Maryland that support this approach are presented.>

An introductory computer science course is developed, much as calculus is a basic course for mathematics and the physical sciences, concerned primarily with theoretical foundations and methodology rather than apprenticeship through applications. In this work, the principles taught in the course are described and an example illustrating them is given.>

The development of four separate, prototype expert systems to aid in software engineering management is described. Given the values for certain metrics, these systems provide interpretations which explain any abnormal patterns of these values during the development of a software project. The four expert systems which solve the same problem, were built using two different approaches to knowledge acquisition, a bottom-up approach and a top-down approach and two different expert system methods, rule-based deduction and frame-based abduction. In a comparison to see which methods might better suit the needs of this field, it was found that the bottom-up approach led to better results that did the top-down approach, and the rule-based deduction systems using simple rules provided more complete and correct solutions than did the frame-based abduction systems.>

An empirical study is presented that investigates hierarchical software system descriptions that are based on measures of cohesion and coupling. The study evaluates the effectiveness of the hierarchical descriptions in identifying error-prone system structure. The measurement of cohesion and coupling is based on intrasystem interaction in terms of software data bindings. The measurement of error-proneness is based on software error data collected from high-level system design through system test; some error data from system operation are also included. The data bindings software analysis and supporting tools are described, followed by the data analysis, interpretations of the results, and some conclusions.>

Experience from a dozen years of analyzing software engineering processes and products is summarized as a set of software engineering and measurement principles that argue for software engineering process models that integrate sound planning and analysis into the construction process. In the TAME (Tailoring A Measurement Environment) project at the University of Maryland, such an improvement-oriented software engineering process model was developed that uses the goal/question/metric paradigm to integrate the constructive and analytic aspects of software development. The model provides a mechanism for formalizing the characterization and planning tasks, controlling and improving projects based on quantitative analysis, learning in a deeper and more systematic way about the software process and product, and feeding the appropriate experience back into the current and future projects. The TAME system is an instantiation of the TAME software engineering process model as an ISEE (integrated software engineering environment). The first in a series of TAME system prototypes has been developed. An assessment of experience with this first limited prototype is presented including a reassessment of its initial architecture.>

This study applies an experimentation methodology to compare three state-of-the-practice software testing techniques: a) code reading by stepwise abstraction, b) functional testing using equivalence partitioning and boundary value analysis, and c) structural testing using 100 percent statement coverage criteria. The study compares the strategies in three aspects of software testing: fault detection effectiveness, fault detection cost, and classes of faults detected. Thirty-two professional programmers and 42 advanced students applied the three techniques to four unit-sized programs in a fractional factorial experimental design. The major results of this study are the following. 1) With the professional programmers, code reading detected more software faults and had a higher fault detection rate than did functional or structural testing, while functional testing detected more faults than did structural testing, but functional and structural testing were not different in fault detection rate. 2) In one advanced student subject group, code reading and functional testing were not different in faults found, but were both superior to structural testing, while in the other advanced student subject group there was no difference among the techniques. 3) With the advanced student subjects, the three techniques were not different in fault detection rate. 4) Number of faults observed, fault detection rate, and total effort in detection depended on the type of software tested. 5) Code reading detected more interface faults than did the other methods. 6) Functional testing detected more control faults than did the other methods.

The Cleanroom software development approach is intended to produce highly reliable software by integrating formal methods for specification and design, nonexecution-based program development, and statistically based independent testing. In an empirical study, 15 three-person teams developed versions of the same software system (800-2300 source lines); ten teams applied Cleanroom, while five applied a more traditional approach. This analysis characterizes the effect of Cleanroom on the delivered product, the software development process, and the developers.

A framework is presented for analyzing most of the experimental work performed in software engineering over the past several years. The framework of experimentation consists of four categories corresponding to phases of the experimentation process: definition, planning, operation, and interpretation. A variety of experiments are described within the framework and their contribution to the software engineering discipline is discussed. Some recommendations for the application of the experimental process in software engineering are included.

The problem of generalizing functional specifications for while loops is considered. This problem occurs frequently when trying to verify that an initialized loop satisfies some functional specification, i.e., produces outputs which are some function of the program inputs. The notion of a valid generalization of a loop specification is defined. A particularly simple valid generalization, a base generalization, is discussed. A property of many commonly occurring while loops, that of being uniformly implemented, is defined. A technique is presented which exploits this property in order to systematically achieve a valid generalization of the loop specification. Two classes of uniformly implemented loops that are particularly susceptible to this form of analysis are defined and discussed. The use of the proposed technique is illustrated with a number of applications. Finally, an implication of the concept of uniform loop implementation for the validation of the obtained generalization is explained.

This paper describes research conducted by the Software Engineering Laboratory (SEL) on the use of dynamic variables as a tool to monitor software development. The intent of the project is to identify project independent measures which may be used in a management tool for monitoring software development. This study examines several Fortran projects with similar profiles. The staff was experienced in developing these types of projects. The projects developed serve similar functions. Because these projects are similar we believe some underlying relationships exist that are invariant between the projects. These relationships, once well defined, may be used to compare the development of different projects to determine whether they are evolving the same way previous projects in this environment evolved.

This paper examines the use of cluster analysis as a tool for system modularization. Several clustering techniques are discussed and used on two medium-size systems and a group of small projects. The small projects are presented because they provide examples (that will fit into a paper) of certain types of phenomena. Data bindings between the routines of the system provide the basis for the bindings. It appears that the clustering of data bindings provides a meaningful view of system modularization.

An effective data collection methodology for evaluating software development methodologies was applied to five different software development projects. Results and data from three of the projects are presented. Goals of the data collection included characterizing changes, errors, projects, and programmers, identifying effective error detection and correction techniques, and investigating ripple effects.

An effective data collection method for evaluating software development methodologies and for studying the software development process is described. The method uses goal-directed data collection to evaluate methodologies with respect to the claims made for them. Such claims are used as a basis for defining the goals of the data collection, establishing a list of questions of interest to be answered by data analysis, defining a set of data categorization schemes, and designing a data collection form. The data to be collected are based on the changes made to the software during development, and are obtained when the changes are made. To ensure accuracy of the data, validation is performed concurrently with software development and data collection. Validation is based on interviews with those people supplying the data. Results from using the methodology show that data validation is a necessary part of change data collection. Without it, as much as 50 percent of the data may be erroneous. Feasibility of the data collection methodology was demonstrated by applying it to five different projects in two different environments. The application showed that the methodology was both feasible and useful.

The problem of analyzing an initialized loop and verifying that the program computes some particular function of its inputs is addressed. A heuristic technique for solving these problems is proposed that appears to work well in many commonly occurring cases. The use of the technique is illustrated with a number of applications. An attribute of initialized loops is identified that corresponds to the ``effort'' required to apply this method in a deterministic (i.e., guaranteed to succeed) manner. It is explained that in any case, the success of the proposed heuristic relies on the loop exhibiting a ``reasonable'' form of behavior.

A family of syntactic complexity metrics is defined that generates several metrics commonly occurring in the literature. The paper uses the family to answer some questions about the relationship of these metrics to error-proneness and to each other. Two derived metrics are applied; slope which measures the relative skills of programmers at handling a given level of complexity and r square which is indirectly related to the consistency of performance of the programmer or team. The study suggests that individual differences have a large effect on the significance of results where many individuals are used. When an individual is isolated, better results are obtainable. The metrics can also be used to differentiate between projects on which a methodology was used and those on which it was not.

The desire to predict the effort in developing or explain the quality of software has led to the proposal of several metrics in the literature. As a step toward validating these metrics, the Software Engineering Laboratory has analyzed the Software Science metrics, cyclomatic complexity, and various standard program measures for their relation to 1) effort (including design through acceptance testing), 2) development errors (both discrete and weighted according to the amount of time to locate and frix), and 3) one another. The data investigated are collected from a production Fortran environment and examined across several projects at once, within individual projects and by individual programmers across projects, with three effort reporting accuracy checks demonstrating the need to validate a database. When the data come from individual programmers or certain validated projects, the metrics' correlations with actual effort seem to be strongest. For modules developed entirely by individual programmers, the validity ratios induce a statistically significant ordering of several of the metrics' correlations. When comparing the strongest correlations, neither Software Science's E metric, cyclomatic complexity nor source lines of code appears to relate convincingly better with effort than the others

This paper reports on an experiment in trying to understand an unfamiliar program of some complexity and to record the authors' understanding of it. The goal was to simulate a practicing programmer in a program maintenance environment using the techniques of program design adapted to program understanding and documentation; that is, given a program, a specification and correctness proof were developed for the program. The approach points out the value of correctness proof ideas in guiding the discovery process. Toward this end, a variety of techniques were used: direct cognition for smaller parts, discovering and verifying loop invariants for larger program parts, and functions determined by additional analysis for larger program parts. An indeterminate bounded variable was introduced into the program documentation to summarize the effect of several program variables and simplify the proof of correctness.

A software engineering research study has been undertaken to empirically analyze and compare various software development approaches; its fundamental features and initial findings are presented in this paper. An experiment was designed and conducted to confirm certain suppositions concerning the beneficial effects of a particular disciplined methodology for software development. The disciplined methodology consisted of programming teams employing certain techniques and organizations commonly defined under the umbrella term structured programming. Other programming teams and individual programmers both served as control groups for comparison. The experimentally tested hypotheses involved a number of quantitative, objective, unobtrusive, and automatable measures of programming aspects dealing with the software development process and the developed software product. The experiment's results revealed several programming aspects for which statistically significant differences existed between the disciplined methodology and the control groups. The results were interpreted as confirmation of the original suppositions and evidence in favor of the disciplined methodology. This paper describes the specific features of the experiment; outlines the investigative approach used to plan, execute, and analyze it; reports its immediate results; and interprets them according to intuitions regarding the disciplined methodology.

This paper discusses axiomatic and functional models of the semantics of structured programming. The models are presented together with their respective methodologies for proving program correctness and for deriving correct programs. Examples using these methodologies are given. Finally, the models are compared and contrasted.

This paper recommends the iterative enhancement' technique as a practical means of using a top-down, stepwise refinement approach to software development. This technique begins with a simple initial implementation of a property chosen (skeletal) subproject which is followed by the gradual enhancement of successive implementations in order to build the full implementation. The development and quantitative analysis of a production compiler for the language SIMPL-T is used to demonstrate that the application of iterative enhancement to software development is practical and efficient, encourages the generation of an easily modifiable product, and facilities reliability.

A great deal of interest has developed in structured programming [Dahl, Dijkstra, and Hoare, 1972] during the past few years. This paper is concerned with some experiences obtained in the use of a structured programming language in the computer science curriculum at the University of Maryland. The language used was SIMPL-X [Basili, 1973], a language designed and implemented at the University of Maryland.