Abstract Technical debt (TD) is a metaphor used to refer to the added software system costs acquired from taking shortcuts. Unfortunately, large amounts of TD can lead to serious consequences, and, thus, the management of TD is essential. Due to TD being a relatively new subject of study, many aspects of TD remain ambiguous. Fortunately, Twitter has been proven to hold a wealth of information on many subjects. As such, this survey study aims to gain a better understanding on how interest in TD has evolved over time and how TD is addressed on Twitter. A total of 128,897 TD‐related tweets were scrapped from Twitter and analyzed using a number of proxy measures and Latent Dirichlet Allocation (LDA). The results revealed that interest in TD on Twitter has been generally increasing since the platform's early stages. Furthermore, TD‐related tweets were found to revolve around 11 distinct categories. The TD in games category was discovered to be the most popular category, followed by TD communication and TD repayment. The results highlight that TD is a diverse and overarching topic that contains many potential avenues for further exploration. Software engineering researchers, practitioners, and educators can utilize this study to help steer their TD‐related future efforts.

Quantization has been applied to multiple domains in Deep Neural Networks (DNNs). We propose Depthwise Quantization (DQ) where quantization is applied to a de-composed sub-tensor along the feature axis of weak statis-tical dependence. The feature decomposition leads to an exponential increase in representation capacity with a linear increase in memory and parameter cost. In addition, DQ can be directly applied to existing encoder-decoder frame-works without modification of the DNN architecture. We use DQ in the context of Hierarchical Auto-Encoders and train end-to-end on an image feature representation. We provide an analysis of the cross-correlation between spatial and channel features and propose a decomposition of the image feature representation along the channel axis. The improved performance of the depthwise operator is due to the increased representation capacity from implicit feature decoupling. We evaluate DQ on the likelihood estimation task, where it outperforms the previous state-of-the-art on CIFAR-10, ImageNet-32 and ImageNet-64. We progressively train with increasing image size a single hierarchical model that uses 69% fewer parameters and has faster convergence than the previous work.

Estimating the cost and schedule of agile software projects is critical at an early phase to establish baseline budgets and schedules for the selection of competitive bidders. The challenge is that common agile sizing measures such as story points and user stories are not practical for early estimation as these are often reported after contract award in DoD. This study provides a set of effort and schedule estimation models for agile projects using a sizing measure that is available before proposal evaluation based on data from 36 DoD agile projects. The results suggest that initial software requirements, defined as the sum of functions and external interfaces, is an effective sizing measure for early estimation of effort and schedule of agile projects. The models’ accuracy improves when application domain groups and peak staff are added as inputs.

Context: The Google Play Store allows app developers to respond to user reviews. Existing research shows that response strategies vary considerably. In addition, while responding to reviews can lead to several types of favorable outcomes, not every response leads to success, which we define as increased user ratings.

Multimodal Transformers achieve superior performance in multimodal learning tasks.However, the quadratic complexity of the selfattention mechanism in Transformers limits their deployment in low-resource devices and makes their inference and training computationally expensive.We propose multimodal Sparse Phased Transformer (SPT) to alleviate the problem of self-attention complexity and memory footprint.SPT uses a sampling function to generate a sparse attention matrix and compress a long sequence to a shorter sequence of hidden states.SPT concurrently captures interactions between the hidden states of different modalities at every layer.To further improve the efficiency of our method, we use Layer-wise parameter sharing and Factorized Co-Attention that share parameters between Cross Attention Blocks, with minimal impact on task performance.We evaluate our model with three sentiment analysis datasets and achieve comparable or superior performance compared with the existing methods, with a 90% reduction in the number of parameters.We conclude that (SPT) along with parameter sharing can capture multimodal interactions with reduced model size and improved sample efficiency.

Background. The software architecture recovery method RELAX produces a concern-based architectural view of a software system graphically and textually from that system's source code. The method has been implemented in software which can recover the architecture of systems whose source code is written in Java. Aims. Our aim was to find out whether the availability of architectural views produced by RELAX can help maintainers who are new to a project in becoming productive with development tasks sooner, and how they felt about working in such an environment. Method. We conducted a user study with nine participants. They were subjected to a controlled experiment in which maintenance success and speed with and without access to RELAX recovery results were compared to each other. Results. We have observed that employing architecture views produced by RELAX helped participants reduce time to get started on maintenance tasks by a factor of 5.38 or more. While most participants were unable to finish their tasks within the allotted time when they did not have recovery results available, all of them finished them successfully when they did. Additionally, participants reported that these views were easy to understand, helped them to learn the system's structure and enabled them to compare different versions of the system. Conclusions. Through the speedup to the start of maintenance experienced by the participants as well as in their formed opinions, RELAX has shown itself to be a valuable help that could provide the basis of further tools that specifically support the development process with a focus on maintenance.

This paper presents a preliminary and exploratory study of the relationship of software security-related size, in lines of code, and the software security level, expressed as the vulnerability density. We developed a classifier and employed source code analyzers to determine the proportion of security-related code and the security level of a sample of 162 open source software (OSS) projects, developed in five different programming languages. Based on the information collected, we analyzed the proportion of security-related code and investigated its correlation with the static analysis vulnerability density of the selected projects. We found that the proportion of security size in OSS can vary depending on its type. Enterprise OSS present a median of 22 % of its code related to security, whereas volunteer-based OSS have a 9% median. We did not find a correlation between security-related size and vulnerability density. However, by filtering only volunteer-based OSS projects in the sample, we found a moderate positive correlation between vulnerability density and the proportion of security-related size. Our study results do not support the idea that a larger proportion of security-related code is associated with more security in terms of a lower vulnerability density. Nonetheless, the type of OSS, namely volunteer-based or enterprise, seems to have a significant impact on software security. Volunteer-based OSS presented a smaller proportion of security-related code, but as this proportion increases, so does the vulnerabilities, indicating a worsening on the security level. The phenomenon was not identified in enterprise OSS.

Abstract Transactions have been used in many software sizing methods to measure software functional size and provide a basis for effort estimation. To further investigate the effects that transactions have on software functional size and more accurately estimate project effort using the transactional information, in this paper, we propose a method to identify transactions, as well as their complexity attributes that are influential to software functional size, directly from the source code of Android projects using static and dynamic analyses. The identified transactions are classified into different complexity levels and applied with different weights based on their evaluated complexity levels to distinguish their effects on software functional size. A size metric called sum of weighted transactions (SWT) is proposed to measure software functional size. We calibrate an effort estimation model using SWT based on a data set of 34 open source Android projects, and show the effectiveness of using SWT as a software functional size measure to estimate project effort.

Until recently, communications regarding apps on the iOS App Store have been one-way from users to developers, with developers unable to respond to reviews directly. While studies have shown that responding to reviews improves an app's overall rating and user satisfaction, resource limitations make it so developers can usually only respond to some of the reviews. Although developers' response behavior has been studied, little is known about which features (aspects) of user reviews spur their responses. Motivated by these observations, we investigate a wide range of features that can be extracted from a user review and apply a random forest algorithm and the features it extracts to predict whether developers will respond to that review. We then determine the importance of these features in distinguishing reviews that receive a developer response from those that do not. Through a case study of three popular free-to-download iOS apps, we find that although features such as rating and review length are among the most important features for all apps, each app has its own individual feature importance ranking, indicating that developers assign different feature weights when prioritizing reviews. Our results may help guide research or the development of tools that are more in line with developers' actual response behavior.

Background: Which aspects of an iOS App Store user review motivate developers to respond? Numerous studies have been conducted to extract useful information from reviews, but limited effort has been expended to answer this question.

Repaying all technical debt (TD) present in a system may be unfeasible, as there is typically a shortage in the resources allocated for TD repayment. Therefore, TD prioritization is essential to best allocate such resources to determine which TD items are to be repaid first and which items are to be delayed until later releases. This study conducts a systematic literature review (SLR) to identify and analyze the currently researched TD prioritization approaches. The employed search strategy strove to achieve high completeness through the identification of a quasi-gold standard set, which was used to establish a search string to automatically retrieve papers from select research databases. The application of selection criteria, along with forward and backward snowballing, identified 24 TD prioritization approaches. The analysis of the identified approaches revealed a scarcity of approaches that account for cost, value, and resources constraint and a lack of industry evaluation. Furthermore, this SLR unveils potential gaps in the current TD prioritization research, which future research may explore.

Microservice architecture has been recognized as an important enabler for continuous development of many cloud-based systems. Code generation has been tried in the tool chain of building microservices. However, most existing tools generally do not consider the risks from continuous development.

Software repositories allow multiple developers to iteratively contribute commits, with the intention of improving the system. However, commits can negatively impact software quality, or even cause the software to become uncompilable. Recent studies show that uncompilable commits exist even in high-profile open-source software. Identifying broken code, a potential symptom of careless development, and analyzing how software changes when it becomes uncompilable can shed light on how software quality evolves when developers do not follow best practices. Since comprehensive software quality analysis tools are incapable of analyzing uncompilable commits, there is little insight as to what happens and how quality changes when a commit breaks the compilability. In this paper, starting from an analysis of the software quality metric changes that happen when the project become uncompilable, we explore the purposes of commits and the relations between commit type, size and compilability, analyzed across 68 open-source Java repositories.

Building more secure software is a recent concern for software engineers due to increasing incidences of data breaches and other types of cyber attacks. However, software security, through the introduction of specialized practices in the software development life cycle, leads to an increase in the development cost. Although there are many studies on software cost models, few address the additional costs required to build secure software. We conducted a systematic review in the form of a mapping study to classify and analyze the literature related to the impact of security in software development costs. Our search strategy strove to achieve high completeness by the identification of a quasi-gold-standard set of papers, which we then used to establish a search string and retrieve papers from research databases automatically. The application of inclusion/exclusion criteria resulted in a final set of 54 papers, which were categorized according to the approach to software security cost analysis. Perform Security Review, Apply Threat Modeling, and Perform Security Testing were the three most frequent activities related to cost, and Common Criteria was the most applied standard. We also identified ten approaches to estimating software security costs for development projects; however, their validation remains a challenge, which could be addressed in future studies.

Background: Software projects are facing the need to adopt security practices during the software development life cycle (SDLC). Nevertheless, the amount of effort to be invested in order to achieve a certain level of software security is not clear yet. Aims: The goal of this study is to get an overview of the application of software security practices in the industry and to identify the impact of the introduction of such activities in software development projects in terms of effort/cost. Method: We conducted a survey on a software security group of a professional social network by applying a random sampling strategy to establish a representative set of participants. Results: The questionnaire was fully answered by 110 participants, from the 808 profiles that were invited from the sampling frame. The results show that security practices have been applied thoroughly in the projects and revealed high variability in secure software development effort across the participants' projects. Further research is needed to understand the different professionals' perspectives regarding security effort in projects. As lessons learned, we found that the professional social network offered a demographically diverse sampling frame, but this comes with hurdles that need to be overcome. Conclusions: The experiences of the participants showed that security is a factor that drives effort in software projects, and security practices need to be taken into account when planning software development initiatives. Our findings about the current state of practices and adoptions can help practitioners and researchers in future endeavors.

Prior work on mobile app reviews has demonstrated that user reviews contain a wealth of information and are seen as a potential source of requirements. However, most of the studies done in this area mainly focused on mining and analyzing user reviews from the US App Store, leaving reviews of users from other countries unexplored. In this paper, we seek to understand if the perception of the same apps between users from other countries and that from the US differs through analyzing user reviews. We retrieve 300,643 user reviews of the 15 most downloaded iOS apps of 2018, published directly by Apple, from nine English-speaking countries over the course of 5 months. We manually classify 3,358 reviews into several software quality and improvement factors. We leverage a random forest based algorithm to identify factors that can be used to differentiate reviews between the US and other countries. Our preliminary results show that all countries have some factors that are proportionally inconsistent with the US.

The stakeholders of a system are legitimately interested in whether and how its architecture reflects their respective concerns at each point of its development and maintenance processes. Having such knowledge available at all times would enable them to continually adjust their systems structure at each juncture and reduce the buildup of technical debt that can be hard to reduce once it has persisted over many iterations. Unfortunately, software systems often lack reliable and current documentation about their architecture. In order to remedy this situation, researchers have conceived a number of architectural recovery methods, some of them concern-oriented. However, the design choices forming the bases of most existing recovery methods make it so none of them have a complete set of desirable qualities for the purpose stated above. Tailoring a recovery to a system is either not possible or only through iterative experiments with numeric parameters. Furthermore, limitations in the scalability of the employed recovery algorithms make it prohibitive to apply the existing techniques to large systems. Finally, since several current recovery methods employ nondeterministic sampling, their inconsistent results do not lend themselves well to tracking a systems course over several versions, as needed by its stakeholders. RELAX (RELiable Architecture EXtraction), a new concern based recovery method that uses text classification, addresses these issues efficiently (1) by assembling the overall recovery result from smaller, independent parts, (2) basing it on an algorithm with linear time complexity and (3) being tailorable to the recovery of a single system or a sequence thereof through the selection of meaningfully named, semantic topics. An intuitive and informative architectural visualization rounds out RELAX's contributions. RELAX is illustrated on a number of existing open-source systems and compared to other recovery methods. Index Terms-software architecture, architectural change, software evolution, open source software, architecture recovery, software development management, software maintenance.

Technical Debt (TD) prioritization is the process of deciding which TD items should be repaid first and which items can be endured until later releases. The goal of the process is to maximize the value of the TD repayment with limited resources. Unfortunately, researchers have indicated the scarcity of TD prioritization techniques and limitations in them. To address these limitations, we propose a novel search-based approach for prioritizing TD using a Multi-objective Evolutionary Algorithm (MOEA). The approach indicates which TD items should be repaid to maximize the value of a repayment activity within a specific cost constraint. An empirical evaluation that we performed on 40 Open-Source Software (OSS) systems demonstrated our approach's ability to improve the value of TD repayment by 1,796 over random search. Additionally, a user study that we conducted with developers confirmed the suitability of our approach in industry and its usefulness in improving the value of TD repayment over developers' prioritization by 423.

Despite the importance of software maintainability in the life cycle of software systems, accurate measurement remains difficult to achieve. Previous work has shown how bug reports can be classified by expressed quality concerns which can give insight into maintainability across domains and over time. However, the amount of manual effort required to produce such classifications limits its usage. In this paper, we build a fuzzy classifier with linguistic patterns to automatically map issue summaries into the seven subgroup SQ classifications provided in a software maintainability ontology. We investigate how long it takes to generate a stable set of rules and evaluate the performance of the rule set on both rule generating and nonrule generating projects. The results validate the generalizability of the fuzzy classifier in correctly and automatically identifying the subgroup SQ classifications from given issue summaries. This provides a building block for analyzing project maintainability on a larger scale.

Abstract In model‐based software estimation, using the right training data is a key contributor for making accurate predictions, which is crucial for the success of software projects. This study investigates the use of duration‐based windows and estimation by analogy to calibrate COCOMO and assess their estimation performance. We compare these approaches as well as the use of all available historical data using the COCOMO data set of 341 projects and NASA data set of 93 projects. The results show that timing information exists in the data sets affecting estimation accuracy. Given sufficient data for calibration, using recently completed projects within short durations generates more accurate estimates than retaining all historical data or using k‐nearest neighbors based on estimation by analogy. More training data spanning a long period of time may not lead to improved estimation accuracy. This study offers evidence to support the use of projects completed within recent years for training estimation models.

Abstract For the last 4 years, we have been experimenting with the parallel agile (PA) approach. PA achieves significant schedule compression by leveraging parallelism; large teams of developers can independently and concurrently develop scenarios from initial concept through code. This paper summarizes our experience in defining and evolving PA by applying it to four representative emergent‐technology applications: location‐based advertising, picture sharing, bad driver reporting, and a VR/AR game project. In comparison with the mainstream architected agile process that we had been using on similar systems, the PA process has consistently achieved significant speedups in system development while simultaneously reducing defects. PA uses storyboards and prototypes to define both sunny‐day and rainy‐day scenarios, defines requirements for each use case, and decomposes each use case into a conceptual model‐view‐controller (MVC) pattern. PA also uses code generation from UML models to rapidly construct a domain‐driven microservice architecture at the inception of a project. This microservice architecture is then used to enable prototype code to interact with a live database during requirements definition. PA then uses automatic test case generation from the same UML model. The paper summarizes the overall challenge of software schedule compression, identifies managed parallel development as generally the most powerful but least‐practiced strategy for schedule compression, and summarizes the key elements required to support parallelism. It then summarizes the key techniques for scaling up PA, using a previous million‐line command and control project as an example. We have used MS‐degree graduate student projects to gather productivity data because the university environment has afforded us the opportunity to explore massively parallel development over an extended time period. We are now working with a large company to modernize their main legacy system using the PA methods. The prospective looks good as many of the techniques used (eg, domain models, use case analysis, and MVC decomposition) have been proven to be effective in industry on a wide range of projects for multiple decades. The new techniques (eg, executable domain models, and visual modeling of sprint plans) should also serve to make integration and project management work better on large development efforts.

Background: Researchers oftentimes measure quality metrics only in the changed files when analyzing software evolution over commit-history. This approach is not suitable for compilation and using program analysis techniques that require byte-code. At the same time, compiling the whole software not only is costly but may also leave us with many uncompilable and unanalyzed revisions. Aims: We intend to demonstrate if analyzing changes in a module results in achieving a high compilation ratio and a better understanding of software quality evolution. Method: We conduct a large-scale multi-perspective empirical study on 37838 distinct revisions of the core module of 68 systems across Apache, Google, and Netflix to assess their compilability and identify when the software is uncompilable as a result of a developer's fault. We study the characteristics of uncompilable revisions and analyze compilable ones to understand the impact of developers on software quality. Results: We achieve high compilation ratios: 98.4% for Apache, 99.0% for Google, and 94.3% for Netflix. We identify 303 sequences of uncompile commits and create a model to predict uncompilability based on commit metadata with an F1-score of 0.89 and an AUC of 0.96. We identify statistical differences between the impact of affiliated and external developers of organizations. Conclusions: Focusing on a module results in a more complete and accurate software evolution analysis, reduces the cost and complexity, and facilitates manual inspection.

Background: Use Case Points (UCPs) have been widely used to estimate software size for object-oriented projects. Yet, many research papers criticize the UCPs methodology for not being verified and validated with data, leading to inaccurate size estimates.

Software systems are often developed by many developers who have a varying range of skills and habits. These developers have a big impact on software quality. Understanding how different developers and developer characteristics impact the quality of a software is crucial to properly deploy human resources and help managers improve quality outcomes which is essential for software systems success. Addressing this concern, we conduct a study on how different developers and developer characteristics such as developer seniority in a system, frequency of commits, and interval between commits relate to Technical Debt (TD). We performed a large-scale analysis on 19,088 commits from 38 Apache Java systems and applied multiple statistical analysis tests to evaluate our hypotheses. Our empirical evaluation suggests that developers unequally increase and decrease TD, a developer seniority in a software system and frequency of commits are negatively correlated with the TD the developer induces, and a developer commit interval has a positive correlation with the TD the developer induces.

Sequence and class diagrams are widely used to model the behavioral and structural aspects of a software system. A size metric that is defined automatically countable from sequence and class diagrams boosts both the efficiency and the accuracy of size estimation by producing reproducible software size measurements. To fulfill the purposes, a size metric called Detailed Use Case Points (DUCPs) is proposed based on the information automatically derived from sequence and class diagrams. The automation is largely supported by our proposed user-system interaction model (USIM) that fills the gap between the system abstraction by the sizing model and the metamodels of the UML diagrams. The effectiveness of our proposed size metric in project effort estimation is validated by an empirical study of 22 historical projects.

Beyond the functional requirements of a system, software maintainability is essential for project success. While there exists a large knowledge base of software maintainability, this knowledge is rarely used in open source software due to the large number of developers and inefficiency in identifying quality issues. To effectively utilize the current knowledge base in practice requires a deeper understanding of how problems associated with the different qualities arise and change over time. In this paper, we sample over 6000 real bugs found from several Mozilla products to examine how maintainability is expressed with subgroups of repairability and modifiability. Furthermore, we manually study how these qualities evolve as the products mature, what the root causes of the bugs are for each quality and the impact and dependency of each quality. Our results inform which areas should be focused on to ensure maintainability at different stages of the development and maintenance process.

Background: Software effort estimates are necessary and critical at an early phase for decision makers to establish initial budgets, and in a government context to select the most competitive bidder for a contract. The challenge is that estimated software requirements is the only size information available at this stage, compounded with the newly increasing adoption of agile processes in the US DoD. Aims: The objectives are to improve cost estimation by investigating available sizing measures, and providing practical effort estimation models for agile software development projects during the contract bidding phase or earlier. Method: The analysis explores the effects of independent variables for product size, peak staff, and domain on effort. The empirical data for model calibration is from 20 industrial projects completed recently for the US DoD, among a larger dataset of recent projects using other lifecycle processes. Results: Statistical results showed that initial software requirements is a valid size metric for estimating agile software development effort. Prediction accuracy improves when peak staff and domain are added as inputs to the cost models. Conclusion: These models may be used for estimates of agile projects, and evaluating software development contract cost proposals with inputs available during the bidding phase or earlier.

The increasing pace of change in competition, technology, and complexity of software-intensive systems has increased the demand for rapid, reliable, scalable, and evolvable processes. Agile methods have made significant contributions to speeding up software development, but often encounter problems with reliability, scalability, and evolvability. Over the past 3 years, we have been experimenting with an approach called Resilient Agile (RA), which addresses these problems while also speeding up development by finding enablers for parallel systems engineering, development, and test. This paper summarizes our experience in defining and evolving RA by applying it to three representative emergent-technology applications: Location-Based Advertising, Picture Sharing, and Bad Driver Reporting. In comparison with the mainstream Architected Agile process that we had been using on similar systems, the RA process achieved fewer defects and significant speedups in system development and evolution.

Software developers often work on multiple projects and tasks throughout a work day, which may affect their productivity and quality of work. Knowing how working on several projects at a time affects productivity can improve cost and schedule estimations. It also can provide additional insights for better work scheduling and the development process. We want to achieve a better productivity without losing the benefits of work interruptions and multitasking for developers involved in the process. To understand how the development process can be improved, first, we identify work interruptions that mostly have a negative effect on productivity, second, we need to quantitatively evaluate impact of multitasking (task switching, work context switching) and work interruptions on productivity. In this research we study cross-project multitasking among the developers working on multiple projects in an educational setting. We propose a way to evaluate the number of cross-project interruptions among software developers using self-reported work logs. This paper describes the research that found: a) software developers involved in two or more projects on average spend 17% of their development effort on cross-project interruptions, b) the amount of effort spent on interruptions is overestimated by the G. Weinberg’s heuristic, c) the correlation between the number of projects and effort spent by developers on cross-project interruptions is relatively weak, and d) there is strong correlation between the number of projects and the number of interruptions developers reported.

In open software development environment, a large number of feature requests with mixed quality are often posted by stakeholders and usually managed in issue tracking systems. Thoroughly understanding and analyzing the real intents that feature requests imply is a labor-intensive and challenging task. In this paper, we introduce an approach to understand feature requests automatically. We generate a set of fuzzy rules based on natural language processing techniques that classify each sentence in feature requests into a set of categories: Intent, Explanation, Benefit, Drawback, Example and Trivia. Consequently, the feature requests can be automatically structured based on the classification results. We conduct experiments on 2,112 sentences taken from 602 feature requests of nine popular open source projects. The results show that our method can reach a high performance on classifying sentences from feature requests. Moreover, when applying fuzzy rules on machine learning methods, the performance can be improved significantly.

Links between issue reports and corresponding fix commits are widely used in software maintenance. The quality of links directly affects maintenance costs. Currently, such links are mainly maintained by error-prone manual efforts, which may result in missing links. To tackle this problem, automatic link recovery approaches have been proposed by building traditional classifiers with positive and negative links. However, these traditional classifiers may not perform well due to the inherent characteristics of missing links. Positive links, which can be used to build link recovery model, are quite limited as the result of missing links. Since the construction of negative links depends on the number of positive links in many existing approaches, the available negative links also become restricted. In this paper, we point out that it is better to consider the missing link problem as a model learning problem by using positive and unlabeled data, rather than the construction of traditional classifier. We propose PULink, an approach that constructs the link recovery model with positive and unlabeled links. Our experiment results show that compared to existing state-of-the-art technologies built on traditional classifier, PULink can achieve competitive performance by utilizing only 70% positive links that are used in those approaches.

Developers intend to improve the quality of the software as it evolves. However, as software becomes larger and more complex, those intended actions may lead to unintended consequences. Analyzing change in software quality among different releases overlooks fine-grained changes that each commit introduces. We believe that studying software quality before and after each commit (commit-impact analysis) can reveal a wealth of information about how the software evolves and how each change impacts its quality. In this paper, we explore whether each commit has an impact on the source code, investigate the compilability of each impactful commit, examine how source code changes affect software quality metrics, and study the effectiveness of using a certain metric as software quality indicator. We analyze a total of 19,580 commits from 38 Apache Java software systems to better understand how change occurs, why, and by who.

Sample-based scheduler design has become an emerging research topic for its high scalability and simple scheduling process in today's big data cluster.One major limitation of such design is its lack of global cluster knowledge, which leads to sub-optimal decisions.Some cutting edge schedulers solve this issue by deploying an extra centralized component in the cluster to capture the real-time cluster state and inform all schedulers.However, such solution is with high cost and low scalability.As an alternative, we introduce the Collaborated-Cluster State(CCS) technique in this paper.CCS is a low cost solution that merely harms the scalability of sample-based design, while achieving similar performance gain as ECC.Experiments with Google and Yahoo production trace both show that CCS under most scenarios can keep up with ECC's performance while reducing 87.7% (in Google trace) and 73.9% (in Yahoo trace) of communications.

Software cost estimates become more accurate as more information becomes available, but are needed early for business case analyses, bids, and resource management. Use Case Points satisfy the ability to make software size estimates early in the lifecycle because they only require understanding how an actor will use the system. Though Use Case Points are easy to calculate, they might over-simplify a project's size and lead to inaccurate estimates. The Use Case Points method has also been heavily criticized for its technical and environmental factors, since they were not calibrated and verified with data. COCOMO(R) II possesses a rich knowledge base of factors that were calibrated and verified with data and expert judgment. Calculating and calibrating an effort model on Unified Code Count (UCC)'s project set using COCOMO(R) II's parameters and Use Case Points needed an additional relative difficulty factor for greatly improved effort estimates.

Context: Software maintenance is required to fix defects, adapt to changes in the environment, and meet new or changed user requirements. The effort of these tasks need to be estimated to track progress, manage resources, and make decisions. Most widely used cost models use source lines of code (SLOC) as the software size input measure, due to its quantifiability and high correlation with effort. Estimating the SLOC of a project is very difficult in early stages of the software lifecycle. Function Points (FPs) represents software size by functions or modifications to functions, making them easier to calculate early in the lifecycle for new development projects or maintenance tasks. Several cost estimators use FPs to estimate the SLOC of a project to take advantage of existing cost models. Goal: Through empirical analysis, the authors want to determine whether FPs can effectively estimate maintenance tasks, as a better alternative to using SLOC as a software size metric. Additionally, the authors will demonstrate that FPs to SLOC ratios add uncertainty to effort estimates. Method: The empirical analysis will be run on Unified Code Count (UCC)'s dataset, a software tool maintained by University of Southern California (USC). Results: The analyses found that separating projects adding new functions from those modifying existing functions resulted in improved estimation models using FPs. The effort estimation model for projects adding functions to UCC had high prediction accuracy statistics, but less impressive results for projects modifying existing functions in UCC. The effort estimation accuracy became unsatisfactorily low when using a FPs to SLOC ratio. Conclusions: Cost estimators should not use FPs to SLOC ratios for effort estimation due to low prediction accuracy. FPs is only an effective size measure for a portion of UCC's maintenance tasks - specifically for the projects adding new functions to UCC. Another size measure may need to be considered that might be more effective independently or in conjunction with FPs for all of UCC's maintenance tasks.

Context: Most widely used cost models use source lines of code (SLOC) as the software size input measure, due to its quantifiability and high correlation with effort. Estimating the SLOC of a project is very difficult in early stages of the software lifecycle, especially for software maintenance tasks. Depending on the reuse model being used, one would need to size the existing code that needs modifications and the size of the changes being made in SLOC. Functional size measures, such as Function Points (FPs) and the Software Non-functional Assessment Process (SNAP), have been developed to improve the ability to estimate project size early in the lifecycle for both development and maintenance projects. While FPs represent software size by functions; SNAP complements FPs by sizing non-functional requirements, such as data operations and interface design. Goal: SNAP complements Function Points by sizing non-functional requirements, such as data operations and interface design. Through an empirical analysis, the authors want to determine whether SNAP might be an effective software size measure individually or in conjunction with FPs to improve effort estimation accuracy. Method: The empirical analysis will be run on Unified Code Count (UCC)'s dataset, a software tool maintained by University of Southern California (USC). Results: The analyses found that separating projects adding new functions from those modifying existing functions resulted in improved estimation models using SNAP. The effort estimation model for projects modifying functions in UCC had high prediction accuracy statistics, but less impressive results for projects adding existing functions to UCC. The effort estimation accuracy were satisfactory when using SNAP in conjunction with FPs for both groups of projects. Conclusions: SNAP, indeed, complements FPs in terms of the requirements that are considered and sized. Both size metrics should be treated as individual metrics, but can be used together for acceptably accurate cost models in UCC's development environment.

Open Source Software (OSS) is distributed and maintained collaboratively by developers all over the world. However, frequent personnel turnover and lack of organizational management makes it difficult to capture the actual development effort. Various OSS maintenance effort estimation approaches have been developed to provide a way to understand and estimate development effort. The goal of this study is to identify the current state of art of the existing maintenance effort estimation approaches for OSS. We performed a systematic literature review on the relevant studies published in the period between 2000-2015 by both automatic and manual searches from different sources. We derived a set of keywords from the research questions and established selection criteria to carefully choose the papers to evaluate. 29 out of 3,312 papers were selected based on a well designed selection process. Our results show that the commonly used OSS maintenance effort estimation methods are actual effort estimation and maintenance activity time prediction, the most commonly used metrics and factors for actual effort estimation are source code measurements and people related metrics, the most commonly mentioned activity for maintenance activity time prediction is bug fixing. Accuracy measures and cross validation is used for validating the estimation models. Based on the above findings, we identified the issues in evaluation methods for actual maintenance effort estimations and the needs for quantitative OSS maintenance effort inference from size-related metrics. Meanwhile, we highlighted individual contribution and performance measurement as a novel and promising research area.

Software cost and effort estimation is a necessary step in the software development lifecycle to track progress, manage resources, and negotiate. Though many accepted cost models exist, local calibration results in more accurate estimates. Locally calibrating Unified Code Count (UCC)'s dataset based on COCOMO (Constructive Cost Model)® II helped UCC's development team learn which factors affected the effort, the amount of fixed costs associated with training new personnel and required deliverables, and resulted in a well-fitting effort estimation model. These insights give the development team a better understanding of the environment and where improvements are most necessary and possible.

Open source projects rely on issue tracking systems such as JIRA or online forums to keep track of users' feedback, expectations and requested features. However, since users are not fully aware of existing features, when submitting new feature requests, redundant requests often appear in the new feature list. It is a waste of time and effort for project contributors to manually identify and reject them, especially in complex systems with many features. Our research is aiming to find a suitable solution to identify redundant feature requests in OSS projects. We have conducted a survey on a well-known Open Source community, Hibernate and gathered all of its feature requests up-to-date. Through studying and categorizing the characteristics of these feature requests, we have found that about 37% of the feature requests were rejected and the most common rejection reason was redundancy. Also we have found that it is very expensive to identify and resolve these redundant feature requests. In this paper, we have proposed our solution to automatically identify redundant feature requests through a Feature Tree Model along with a future research agenda.

With respect to system thinking, a T-shaped person is one who has technical depth in at least one aspect of the system's content, and a workable level of understanding of a fair number of the other system aspects. Many pure computer science graduates are strongly I-shaped, with a great deal of depth in software technology, but little understanding of the other disciplines involved in such areas as business, medicine, transportation, or Internets of Things. This leaves them poorly prepared to participate in the increasing numbers of projects involving multi-discipline system thinking, and in strong need of software skills. We have developed and evolved an MS-level software engineering curriculum that enables Computer Science (CS) majors to become considerably more T-shaped than when they entered. It includes courses in software management and economics, human-computer interaction, embedded software systems, systems and software requirements, architecture, and Verification and Validation (V&V), and a two-semester, real-client team project course that gives students experience in applying these skills. We find via feedback on the students' internships and job experiences that they and their employers have high rates of success in job offers and job performance.

With respect to system thinking, a T-shaped person is one who has technical depth in at least one aspect of the system's content, and a workable level of understanding of a fair number of the other system aspects. Many pure computer science graduates are strongly I-shaped, with a great deal of depth in software technology, but little understanding of the other disciplines involved in such areas as business, medicine, transportation, or Internets of Things. This leaves them poorly prepared to participate in the increasing numbers of projects involving multi-discipline system thinking, and in strong need of software skills. We have developed and evolved an MS-level software engineering curriculum that enables CS majors to become considerably more T-shaped than when they entered. It includes courses in software management and economics, human-computer interaction, embedded software systems, systems and software requirements, architecture, and V&V, and a two-semester, real-client team project course that gives students experience in applying these skills. We find via feedback on the students' internships and job experiences that they and their employers have high rates of success in job offers and job performance.

This tutorial introduces the ICSM as a means of addressing the need for more flexible and responsive systems engineering. It describes the principles that guide the model, and leads attendees through a product development case study. Attendees will receive a copy of the book The Incremental Commitment Spiral Model: Principles and Practices for Successful Systems and Software.

Real-client, team project courses provide excellent opportunities for performing empirical research in software engineering (SE). Compared to empirical research on large, multi-year SE projects, a course with several team projects per year is the SE research equivalent of the fruit fly in species evolution research. Although their predictive power for large-project SE is more suggestive than definitive, the research results generally provide useful contributions to human knowledge in the SE area.

Context: An effort estimation model with more than 20 parameters is not very useful at early conceptual phase if you don't have a logical approach for specifying the input values.

This paper summarizes several iterations in developing a compact set of four key principles for successful systems engineering, which are 1) Stakeholder Value-Based Guidance 2) Incremental Commitment and Accountability 3) Concurrent Multidiscipline Engineering, and 4) Evidence- and Risk-based Decisions. It provides a rationale for the principles, including short example case studies of failed projects that did not apply the principles, and of successful projects that did. It will compare the principles with other sets of principles such as the Lean Systems Engineering and the Hitchins set of principles for successful systems and systems engineering, and indicate how the principles will help projects and organizations cope with increasing needs for process diversity and change.

The Incremental Commitment Spiral Model (ICSM) presented in an ICSSP 2014 tutorial includes several decision milestones at which evidence of the feasibility of the proposed process is evaluated, and at which the stakeholders decide whether to proceed with it or to change course, based on the risk of proceeding with the proposed process. This generates a large number of potential processes, but we have found risk patterns that provide selection criteria for a set of common cases for at least the initial process.

This paper summarizes the Incremental Commitment Spiral Model (ICSM), a process model generator that enables organizations to determine which process model, or combination of models, best fits the needs of each system.

The phenomenon called Incremental Development Productivity Decline (IDPD) is presumed to be present in all incremental soft-ware projects to some extent. COCOMO II is a popular parametric cost estimation model that has not yet been adapted to account for the challenges that IDPD poses to cost estimation. Instead, its cost driver and scale factors stay constant throughout the increments of a project. While a simple response could be to make these parameters variable per increment, questions are raised as to whether the existing parameters are enough to predict the behavior of an incrementally developed project even in that case. Individual COCOMO II parameters are evaluated with regard to their development over the course of increments and how they influence IDPD. The reverse is also done. In light of data collected in recent experimental projects, additional new variable parameters that either extend COCOMO II or could stand on their own are proposed.

This research paper expands on a previously introduced phenomenon called Incremental Development Productivity Decline (IDPD) that is presumed to be present in all incremental software projects to some extent. Incremental models are now being used by many organizations in order to reduce development risks. Incremental development has become the most common method of software development. Therefore its characteristics inevitably influence the productivity of projects. Based on their observed IDPD, incrementally developed projects are split into several major IDPD categories. Different ways of measuring productivity are presented and evaluated in order to come to a definition or set of definitions that is suitable to these categories of projects. Data has been collected and analyzed, indicating the degree of IDPD associated with each category. Several hypotheses have undergone preliminary evaluations regarding the existence, stability and category-dependence of IDPD with encouraging results. Further data collection and hypothesis testing is underway.

As an approach to develop suitable development processes for software projects, Software Process Selection, Tailoring and Composition (SP-STC) attract lots of attention from both industry and academia. However, without effective guidelines, how to do SP-STC often remains a mystery. This special panel aims to 1) initiate a discussion on the current research status of SP-STC, 2) identify main challenges of SP-STC and possible solutions, and 3) work out a research agenda for future work.

Which are the consequences of Lehman's Laws of Software Evolution for the productivity of incrementally developed projects? The concept of Incremental Development Productivity Decline (IDPD), which deals with how the productivity of incrementally developed software develops over its increments, is introduced. It is explained how Lehman's Laws of Software Evolution apply to it and how maintenance and reuse are relevant to both. Every Law of Software Evolution is discussed individually from a qualitative standpoint with regard to whether it could be a cause of IDPD. After that discussion, the overall situation is examined in light of how different courses of action cause which laws to apply different degrees of effects.

Risk identification, management, and mitigation are essential to the success of any software development projects. At the University of Southern California (USC), CSCI577ab is a graduate level software engineering course sequence that teaches the best software engineering practices, and allows students to apply the learned knowledge in developing real-client projects. This paper analyzes the risks encountered by teams, identifies the relationships between effective risk analysis and the success of the project, and discusses how risk patterns determine the student's course of action. This paper also reports the top risks of software development in the software engineering class.

The term “Technical Debt” was coined over 20 years ago by Ward Cunningham in a 1992 OOPSL A experience report to de scribe the tr ade-offs between delivering the most appropriate — albeit likely immature — product, in the shortest time possible. Since then the repercussions of going into “technical debt” have become more visible, yet not necessarily more broadly understood. This panel will bring together practitioners to discuss and debate strategies for debt relief.

Accelerating development schedules is increasingly important in a competitive world. Reduced time-to-market is a key response to competitive threats in the commercial sphere, and rapid response in deploying military systems may save lives in a geopolitical environment characterized by rapidly emerging and ever-changing physical threats. Agile/lean development methodologies show promise in providing the desired schedule acceleration, but it can be difficult for planners to determine the effects of these factors on schedule duration, and to make appropriate choices to optimize project performance. The Constructive Rapid Application Development Model (CORADMO) attempts to quantify the effects of key schedule drivers, and thus enable planners to estimate the relative schedule that will result from varying these parameters.

Selecting system and software requirements to implement in a particular product or release is a challenging decision problem. Business stakeholders strive to maximize return on investment by selecting the most valuable requirements for implementation. Deciding on the requirements to be selected entails a great deal of communication and coordination amongst the stakeholders to ascertain the priorities of the individual requirements. The prioritized requirements aid in the planning and sequencing of implementation activities associated with the software system and provides a basis of a prioritized backlog from which the requirements can be ‘pulled’ for development. Changing business priorities may require a complete reprioritization of the backlog, leading to wasted effort. Individual change requests and new requirements need to be prioritized and inserted into the correct location in the backlog requiring high communication overhead. In this paper we summarize a two-step prioritization approach using a decision theoretic model to prioritize system and software requirements that alleviates these concerns. The system is initially decomposed into high-level Minimal Marketable Features (MMFs) and each MMF is further decomposed into low-level requirements. The MMFs are prioritized against the business goals of the organization and the low-level requirements with respect to ease of realization and business value. The priorities of the individual requirements are influenced by that of the MMFs they belong to. This two-step approach serves as an important prelude for a dynamically prioritizable product backlog. In this paper we present a proof-of-concept of having implemented this approach with 24 real-client student project teams at the Software Engineering project course at the University of Southern California.

Most of the current testing strategies treat all aspects of software equally important in a value-neutral way; this becomes more risky when the testing resources are limited. Our goal in this case study aims at improving the testing cost-effectiveness of an industrial scenario testing process under time constraints. We proposed a value-based testing prioritization strategy which allows tests to be ranked by how well the tests can reduce risk exposure. Combining this with the tests’ relative costs enables them to be prioritized in terms of return on investment (ROI) or risk reduction leverage (RRL). Besides, a new metric Average Percentage of Business Importance Earned (APBIE) is proposed to measure how quickly the testing can reduce the quality uncertainty and earn the relative business importance of the system under test (SUT).The results from one case study to prioritize operational testing scenarios in Galorath Inc. show that our method can improve the testing cost-effectiveness in terms of APBIE.

Is quality really free? Often, but not in many situations. Are organizations getting the best value for their investments in reliability? Frequently not. If different stakeholders are relying on the system and software for different properties, is there a single reliability metric to manage to? Frequently not. Is reliability affected by decisions on other system and software ilities? Often quite seriously. This talk will review a number of data sources, case studies, calibrated models, and recent research on systems and software ility tradeoffs to suggest ways that can help improve an organization's return on investments in more reliable systems and software.

Background: Given information on just a few prior projects, how do we learn the best and fewest changes for current projects? Aim: To conduct a case study comparing two ways to recommend project changes. 1) Data farmers use Monte Carlo sampling to survey and summarize the space of possible outcomes. 2) Case-based reasoners (CBR) explore the neighborhood around test instances. Method: We applied a state-of-the data farmer (SEESAW) and a CBR tool ()'V2) to software project data. Results: CBR with )'V2 was more effective than SEESAW's data farming for learning best and recommended project changes, effectively reducing runtime, effort, and defects. Further, CBR with )'V2 was comparably easier to build, maintain, and apply in novel domains, especially on noisy data sets. Conclusion: Use CBR tools like )'V2 when data are scarce or noisy or when project data cannot be expressed in the required form of a data farmer. Future Work: This study applied our own CBR tool to several small data sets. Future work could apply other CBR tools and data farmers to other data (perhaps to explore other goals such as, say, minimizing maintenance effort).

Team synchronization and stabilization are essential - especially for large software projects. However, often little is done to assess and reduce the uncertainties and knowledge gaps that exist within the project. As the project progresses through its life cycle, the team can gain more information about the project and team's capabilities. These necessary data can be obtained through performing assessments on the team and project. As these assessments procedures are often complex, discouraging, and difficult to analyze, an effective framework and tool support can greatly enhance the process. Hence, with improved assessment methods, software project teams can quickly gather the necessary data, determine the actions to improve performance, and result in an improved project outcome in the end. The COnstructive Team Improvement Process MOdel (COTIPMO) is a framework developed to effectively improve team synchronization and stabilization as well as project effort estimation and scoping by enabling software development teams to quickly track project progress, continuously assess team performance, and make adjustments to the project estimates as necessary.

Avoiding a major source of system and software project failures by finding more non-technical-user friendly methods of system definition and evolution has been a significant challenge. Five generations of the WinWin negotiation framework have improved such capabilities, but even the latest WikiWinWin toolset has encountered problems with non-technical stakeholder usage. With the advent of social networking and popularity of Facebook and Gmail, we have developed a radically different way for collaborative requirements management and negotiations. The new avatar of the WinWin framework called `Winbook' is based on the social networking paradigm, similar to Facebook and content organization using color coded labels, similar to Gmail. Initial usage results on 14 small projects involving non-technical stakeholders have shown profound implications on the way requirements are negotiated and used through the system and software definition and development processes. Winbook has also been adopted as part of a project to bridge requirements and architecting for a major US government organization.

There are usually more requirements than feasible in a given schedule. Thus, it's imperative to be able to choose the most valuable ones for implementation to ensure the delivery of a high value software system. There are myriad requirements prioritization frameworks and selecting the most appropriate one is a decision problem in its own right. In this paper we present our approach in selecting the most appropriate value based requirements prioritization framework as per the requirements of our stakeholders. Based on our analysis a single framework was selected, validated by requirements engineers and project managers and deployed for company-wide use by a major IT player in India.

SUMMARY For a successful software project, acceptable quality must be achieved within an acceptable cost, demonstrating business value to customers and satisfactorily meeting delivery timeliness. Testing serves as the most widely used approaches to determine that the intended functionalities are performed correctly and achieve the desired level of services; however, it is also a labor‐intensive and expensive process during the whole software life cycle. Most current testing processes are often technique‐centered, rather than organized to maximize business value. In this article, we extend and elaborate the ‘4+1’ theoretical lenses of Value‐based Software Engineering (VBSE) framework in the software testing process; propose a multi‐objective feature prioritization strategy for testing planning and controlling, which aligns the internal testing process with value objectives coming from customers and markets. Our case study in a real‐life business project shows that this method allows reasoning about the software testing process in different dimensions: it helps to manage the testing process effectively and efficiently, provides information for continuous internal software process improvement, and increases customer satisfaction, which makes winners of all success‐critical stakeholders (SCSs) in the software testing process. Copyright © 2010 John Wiley & Sons, Ltd.

The Software Engineering Body of Knowledge (SWEBOK), published in 2004, and now under revision, has influenced many software engineering graduate programs worldwide. In 2009, guidelines were published for graduate programs in software engineering (GSWE2009). GSWE2009, now sponsored by both the IEEE Computer Society and the Association for Computing Machinery, strongly rely on the SWEBOK but also recommends specific systems engineering knowledge for students to master. Today, an international team is creating a rigorous Systems Engineering Body of Knowledge (SEBoK) with the help of the IEEE Computer Society and the International Council on Systems Engineering and other professional societies. As it matures, the SEBoK should influence future versions of GSWE2009 and graduate program curricula worldwide. This panel will examine the influence of bodies of knowledge on both the creation of new graduate software engineering programs and the evolution of existing ones.

Our two-semester USC core software engineering project course CS577ab devotes its first semester to having students learn and do systems engineering on a real-client project. This requires a good deal of just-in-time lectures, tutorials, and homework to prepare the students, and feedback in terms of mentoring, artifact grading, and live milestone reviews to help them succeed. This paper provides some initial motivation and context; discusses our approach to introduce systems engineering into software engineering relative to that in the GSwE 2009 curriculum guidelines, SEBOK draft 2010, and SWEBOK 2004; describes the course practices during the systems engineering and software engineering semesters; and summarizes the project results and conclusions.

An empirical study is provided on teaching Verification & Validation (V&V) process practice in a real-client graduate level software engineering course which makes students and researchers mutual winners. From our observation and experiences during the course, on the education side, several reflection-in-action techniques are used to educate and train students. These include inspections, architecture review boards, grading criteria, monitoring of their quality management plans, student critiques of their project experiences, and client evaluations. On the research side, students' feedback, evaluation, and critiques provide not only previous empirical evidence for the researchers' research proposal, but also great opportunities to refine their research methods from lessons learned from the course, and in turn to improve the course quality.

The article of record as published may be found at http://dx.doi.org/10.1109/ESEM.2011.56

This paper is a contribution to the Impact Project in the area of software resource estimation. The objective of the Impact Project has been to analyze the impact of software engineering research investments on software engineering practice. The paper begins by summarizing the motivation and context for analyzing software resource estimation; and by summarizing the study's purpose, scope, and approach. The approach includes analyses of the literature; interviews of leading software resource estimation researchers, practitioners, and users; and value/impact surveys of estimators and users. The study concludes that research in software resource estimation has had a significant impact on the practice of software engineering, but also faces significant challenges in addressing likely future software trends.

The increasingly rapid pace of change in technology, competition, and IT demand have made evolutionary development an attractive alternative to traditional software and systems engineering (SSE) processes. There are several forms of evolutionary development, and there is no one-size-fits-all SSE approach that is best for all situations. For rapid-fielding situations, an easiest-first, get something working, evolutionary SSE approach is best. But for enduring systems, an easiest-first evolutionary SSE approach is likely to produce an unscalable system whose architecture is incompatible with achieving high levels of safety and security. Also, evolutionary development requires much higher sustained levels of systems engineering (SE) effort, earlier and continuous integration and test, pro-active approaches to address sources of system change, greater levels of concurrent engineering, and achievement reviews based on evidence of feasibility vs. evidence of plans, activity, and system descriptions. Many traditional acquisition practices are incompatible with effective SE of evolutionary development. These include assumptions that full-capability requirements can be specified up front along with associated full-capability plans, budgets, schedules, work breakdown structures, and earned-value management targets; that most systems engineers can be dismissed after PDR; and that all forms of requirements change or "creep" should be discouraged. The study also found that other inhibitors to effective SE need to be addressed, such as underbudgeting (SE is the first victim of inadequate budgets); contracting provisions emphasizing functional definition before addressal of key performance parameters; and management temptations to show rapid progress on easy initial increments while deferring the hard parts until later increments.

Today's and tomorrow's complex, interdependent, dynamic systems require richer process principles than the simplistic principles in the Agile Manifesto or in simplistic sequential waterfall or Vee models. The resulting principles should capitalize on the strengths of these while avoiding their weaknesses.

As a new contribution to Value-based V&V process development, a systematic and multi-criteria process is proposed to quantitatively determine the Value-based V&V artifact priority that reviewers can follow for their reviews. This process enables reviewers to prioritize artifacts to be reviewed in a more cost-effective way based on more sophisticated and comprehensive factors, such as importance, quality risks, dependency and cost of V&V investments. Some qualitative and quantitative evidence is provided from a comparative experiment with 22 real-client e-services projects over two years of a graduate software engineering team-project course. It shows that the value-based artifact prioritization enabled reviewers to better focus on artifacts with high importance and risks, to capture issues with high impact in a timely manner, and to improve the cost-effectiveness of reviews.

The definitive version can be found at : http://onlinelibrary.wiley.com/ Copyright Wiley [Full text of this article is not available in the UHRA]

The wide variety of software-intensive systems needed to support the new horizons of evolving technology, system and software complexity, high dependability, global interoperability, emergent requirements, and adaptability to rapid change make traditional and current one-size-fits-all process models infeasible. This tutorial presents the process framework, principles, practices, and case studies for a new model developed and being used to address these challenges. It has a series of risk-driven decision points that enable projects to converge on whatever combination of agile, plan-driven, formal, legacy-oriented, reuse-oriented, or adaptive processes that best fit a project's situation. The tutorial discusses the decision table for common special cases; exit ramps for terminating non-viable projects; support of concurrent engineering of requirements, solutions and plans; and evidence-based commitment milestones for synchronizing the concurrent engineering. The tutorial will include case studies and exercises for participants' practice and discussion.

Accurate software cost and schedule estimations are essential especially for large software projects. However, once the required efforts have been estimated, little is done to recalibrate and reduce the uncertainty of the initial estimates. To address this problem, we have developed and used a framework to continuously monitor the software project progress and readjust the estimated effort utilizing the Constructive Cost Model II (COCOMO II) and the Unified CodeCount Tool developed by the University of Southern California (USC). As a software project progresses, we gain more information about the project itself, which can then be used to assess and re-estimate the effort required to complete the project. With more accurate estimations and less uncertainties, the quality and goal of project outcome can be assured within the available resources. The paper thus also provides and analyzes empirical data on how projects evolve within the familiar software "cone of uncertainty".

This paper describes a controlled experiment of student programmers performing maintenance tasks on a C++ program. The goal of the study is to assess the maintenance size, effort, and effort distribution of three different maintenance types and to describe estimation models to predict the programmer's effort on maintenance tasks. The results of our study suggest that corrective maintenance is much less productive than enhancive and reductive maintenance. Our study also confirms the previous results which conclude that corrective and reductive maintenance requires large proportions of effort on program comprehension activity. Moreover, the best effort model we obtained from fitting the experiment data can estimate the time of 79% of the programmers with the error of 30% or less.

At University of Southern California (USC), CSCI577ab is a graduate software engineering course that teaches best software engineering practices and allows students to apply the learned knowledge in developing real-client projects. The class is used as an experimental test-bed to deploy various research tools and approaches for validation of new methods and tools. Various research data have been collected as partial basis for twelve PhD dissertations. This paper reports how research and education are integrated via project experiments and how the results strengthen future educational experiences.

In an investigating study to trace the productivity changes of a commercial software project, which uses incremental and iterative development model, we've found evidence that attributes such as staffing stability, design compatibility/ adaptability to incremental and iterative development, and integration and testing would have significant impact on modifying the productivity outcome - either positive or negative. In this report, we will present an empirical analysis to review, evaluate, and discuss these influential attributes in regard of their correlations with productivity in incremental and iterative software development. We also hope that our approach and results would contribute to initiate more research in this subject area.

Before performing drastic changes to a project, it is worthwhile to thoroughly explore the available options within the current structure of a project. An alternative to drastic change are internal changes that adjust current options within a software project. In this paper, we show that the effects of numerous internal changes can out-weigh the effects of drastic changes. That is, the benefits of drastic change can often be achieved without disrupting a project. The key to our technique is SEESAW, a novel stochastic stability tool that (a) considers a very large set of minor changes using stochastic sampling; and (b) carefully selects the right combination of effective minor changes. Our results show, using SEESAW, project managers have more project improvement options than they currently realize. This result should be welcome news to managers struggling to maintain control and continuity over their project in the face of multiple demands for drastic change.

Nancy Mead has been expounding the virtues of requirements prioritization, especially in the area of information security, for over a decade. She advocates avoiding purely informal ad-hoc approaches and suggests a prioritization strategy based on both cost of implementing requirements and the expected value if implemented. Due to critical schedule and resource constraints, we have found that requirements prioritization is a key risk management practice for real project for real clients based courses. This work describes our experiences applying Nancy Meadpsilas requirement prioritization works within such courses and how we educate students on the importance of utilizing this for managing requirements implementation risk in student projects within three universities for both graduate and undergraduates.

Building cost estimation models is often considered a search problem in which the solver should return an optimal solution satisfying an objective function. This solution also needs to meet certain constraints. For example, a solution for the estimates coefficients of COCOMO models must be non-negative. In this research, we introduce a constrained regression technique that uses objective functions and constraints to estimate the coefficients of the COCOMO models. To access the performance of the proposed technique, we run a cross-validation procedure and compare the prediction accuracy from different approaches such as least squares, stepwise, Lasso, and Ridge regression. Our result suggests that the regression model that minimizes the sum of relative errors and imposes non-negative coefficients is a favorable technique for calibrating the COCOMO model parameters.

Effort distribution by phase or activity is an important but often overlooked aspect compared to other steps in the cost estimation process. Poor effort allocation is among the major root causes of rework due to insufficiently resourced early activities. This paper provides results of an empirical study on phase effort distribution data of 75 industry projects, from the China Software Benchmarking Standard Group (CSBSG) database. The phase effort distribution patterns and variation sources are presented, and analysis results show some consistency in effects of software size and team size on code and test phase distribution variations, and some considerable deviations in requirements, design, and transition phases, compared with recommendations in the COCOMO model. Finally, this paper discusses the major findings and threats to validity and presents general guidelines in directing effort allocation. Empirical findings from this study are beneficial for stimulating discussions and debates to improve cost estimation and benchmarking practices.

In 2005, we studied the development effort and effect of quality comparisons between software development with Fagan's inspection and pair development. Three experiments were conducted in Thailand: two classroom experiments and one industry experiment. We found that in the classroom experiments, the pair development group had less average development effort than the inspection group with the same or higher level of quality. The industry experiment's result showed pair development to have a bit more effort but about 40% fewer major defects. However, since this set of experiments was conducted in Thailand, the results may be different if we conducted the experiment in other countries due to the impact of cultural differences. To investigate this we conducted another experiment with computer science graduate students at USC in Fall 2006. Unfortunately, the majority of the graduate students who participated in the experiment were from India, a country in which the culture is not much different from Thailand. As a result, we cannot compare the impact of cultural differences in this paper. However, the results showed that the experiment can be replicated in other countries where the cultures are similar.

Adoption of advanced automated SE (ASE) tools would be favored if a business case could be made that these tools are more valuable than alternate methods. In theory, software prediction models can be used to make that case. In practice, this is complicated by the "local tuning" problem. Normally, predictors for software effort and defects and threat use local data to tune their predictions. Such local tuning data is often unavailable.

Software systems today are composed from prefabricated commercial components and connectors that provide complex functionality and engage in complex interactions. Unfortunately, because of the distinct assumptions made by developers of these products, successfully integrating them into a software system can be complicated, often causing budget and schedule overruns. A number of integration risks can often be resolved by selecting the 'right' set of COTS components and connectors that can be integrated with minimal effort. In this paper we describe a framework for selecting COTS software components and connectors ensuring their interoperability in software-intensive systems. Our framework is built upon standard definitions of both COTS components and connectors and is intended for use by architects and developers during the design phase of a software system. We highlight the utility of our framework using a challenging example from the data-intensive systems domain. Our preliminary experience in using the framework indicates an increase in interoperability assessment productivity by 50% and accuracy by 20%.

Summary form only given. Much of software engineering is taught and practiced in a value-neutral context, in which every requirement, use case, object, test case, and defect is equally important. Too often, students learn that some of their stakeholders' value considerations are more important than others by failing to consider this on the job and suffering the consequences. The recent book, Value-Based Software Engineering (S. Biffl et al., eds., Springer, 2005) sets out the agenda of the value-based software engineering community. It is to integrate value considerations into traditional software engineering principles and practices for use in software engineers' education and daily work. We have been pursuing this agenda in a research project called "A Value-Based Science of Design", within the NSF Science of Design program. This paper addresses the nature of "value" in a software engineering context; present an initial theory and process for performing value-based software engineering; present example value-based techniques for requirements engineering, design, development, and test; and discuss experiences in incorporating value-based software engineering in individual-assignment and team project courses.

The development of games in aid of improving and enriching a student’s learning experience is again on the rise. The beer game [6] in the field of system dynamics was developed to instill the key principles of production and distribution. SimSE [5] provides a simulated game for its players to take on the role of a project manager, and experience the fundamentals of software engineering through cause-effect models. In this paper we present an initial design of SimVBSE as a game for students to better understand value-based software engineering [1], and its underlying theory [3].

George Santayana's statement, "Those who cannot remember the past are condemned to repeat it," is only half true. The past also includes successful histories. If you haven't been made aware of them, you're often condemned not to repeat their successes.In a rapidly expanding field such as software engineering, this happens a lot. Extensive studies of many software projects such as the Standish Reports offer convincing evidence that many projects fail to repeat past successes.This paper tries to identify at least some of the major past software experiences that were well worth repeating, and some that were not. It also tries to identify underlying phenomena influencing the evolution of software engineering practices that have at least helped the author appreciate how our field has gotten to where it has been and where it is.A counterpart Santayana-like statement about the past and future might say, "In an era of rapid change, those who repeat the past are condemned to a bleak future." (Think about the dinosaurs, and think carefully about software engineering maturity models that emphasize repeatability.)This paper also tries to identify some of the major sources of change that will affect software engineering practices in the next couple of decades, and identifies some strategies for assessing and adapting to these sources of change. It also makes some first steps towards distinguishing relatively timeless software engineering principles that are risky not to repeat, and conditions of change under which aging practices will become increasingly risky to repeat.

Cost, schedule and quality are highly correlated factors in software development. They basically form three sides of the same triangle. Beyond a certain point (the "Quality is Free" point), it is difficult to increase the quality without increasing either cost or schedule or both for the software under development. As products and applications mature, users expect higher quality products. They want IT organizations to be responsible and accountable for the quality claims made by the product marketing teams. In the last couple decades, much software engineering research has focussed on standards, methodologies and techniques for improving software quality, measuring software quality and software quality assurance. Most of this research is focused on the internal/development view of quality. More recent studies done in conjunction with the marketing groups have made attempts to understand the customer view of quality. All of these different ongoing activities to understand quality from the various perspectives have made the field even more enriching and exciting. The Fourth Workshop on Software Quality aims to bring together academic, industrial and commercial communities interested in software quality topics to discuss the different technologies being defined and used in the software quality area.

Commercial organizations increasingly need software processes sensitive to business value, quick to apply, and capable of early analysis for subprocess consistency and compatibility. This paper presents experience in applying a lightweight synthesis of a Value-Based Software Quality Achievement (VBSQA) process and an Object-Petri-Net-based process model (called VBSQA-OPN) to achieve a manager-satisfactory process for software quality achievement in an on-going ERP software project in China. The results confirmed that 1) the application of value-based approaches was inherently better than value-neutral approaches adopted by most ERP software projects; 2) the VBSQA-OPN model provided project managers with a synchronization and stabilization framework for process activities, success-critical stakeholders and their value propositions; 3) process visualization and simulation tools significantly increased management visibility and controllability for the success of software project.

Most risk analysis tools and techniques require the user to enter a good deal of information before they can provide useful diagnoses. In this paper, we describe an approach to enable the user to obtain a COTS glue code integration risk analysis with no inputs other than the set of glue code cost drivers the user submits to get a glue code integration effort estimate with the COnstructive COTS integration cost estimation (COCOTS) tool. The risk assessment approach is built on a knowledge base with 24 risk identification rules and a 3-level risk probability weighting scheme obtained from an expert Delphi analysis. Each risk rule is defined as one critical combination of two COCOTS cost drivers that may cause certain undesired outcome if they are both rated at their worst case ratings. The 3-level nonlinear risk weighting scheme represents the relative probability of risk occurring with respect to the individual cost driver ratings from the input. Further, to determine the relative risk impact, we use the productivity range of each cost driver in the risky combination to reflect the cost consequence of risk occurring. We also develop a prototype called COCOTS Risk Analyzer to automate our risk assessment method. The evaluation of our approach shows that it has done an effective job of estimating the relative risk levels of both small USC e-services and large industry COTS-based applications.

Commercial, public service, and national security organizations are finding it increasingly attractive to integrate component system capabilities from many different best of breed sources to achieve their objectives. In doing so, they find that they are increasingly dependent on software to integrate the systems and to rapidly adapt them in response to competitive opportunities or threats, new technologies, or new organizational priorities. The resulting software-intensive systems of systems (SISOS) offer significant benefits, but come with significant new types of risks, such as simultaneous satisfaction of multiple stakeholders and quality attributes, and integration and rapid adaptation of multiple heterogeneous software products and COTS products. This tutorial provides software engineers, managers and researchers with an understanding of and emerging capabilities for practicing software engineering in the very large. It identifies the major opportunities and risks involved in software- intensive systems of systems, presents experience-based techniques for realizing the opportunities and mitigating the risks. It provides extensions of the risk-driven spiral model that are being used for SISOS development processes, and provides case study exercises to give participants experience in applying the techniques in representative situations.

To develop software on time and within budget is not good enough if the software produced is full of defects. As the software market matures, users want to be assured of quality. They no longer accept claims made by the IT department at face value, but expect concrete demonstrations of quality. Software stakeholders are also demanding higher quality software than ever before. In recent years, much software engineering research has focussed on standards, methodologies and techniques for improving software quality, measuring software quality and software quality assurance. Most of this research is focused on an internal view of quality whereas few measures of the customer view of quality exist. The Third Workshop on Software Quality aims to bring together academic, industrial and commercial communities interested in software quality topics to discuss the different technologies being defined and used in the software quality area.

Rapid change and increasing software criticality drive successful development and acquisition organizations to balance the agility and discipline of their key processes. The emergence of agile methods in the software community is raising the expectations of customers and management, but the methods have shortfalls and their compatibility with traditional plan-driven methods such as those represented by CMMI, ISO-15288, and UK-DefStan-00-55 is largely unexplored. This paper pragmatically examines the aspects of agile and plan-driven methods and provides an approach to balancing through examples and case studies.

Workshop Summary The purpose of this workshop is to present an in-depth overview of USC CS577, and to start the processes of: • eliciting from faculty at a variety of colleges and universities the different needs and resource constraints to which they would be subject were they to port CS577 to their home institutions • planning the types of adjustments that can be made to CS577 to meet various sets of constraints

The field of software economics seeks to develop technical theories, guidelines, and practices of software development based on sound, established, and emerging models of value and value-creation---adapted to the domain of software development as necessary. The premise of the field is that software development is an ongoing investment activity---in which developers and managers continually make investment decisions requiring the expenditure of valuable resources, such as time, talent, and money. The overriding aim of this activity is to maximize the value added subject to an equitable distribution among the participating stakeholders. The goal of the tutorial is to expose the audience to this line of thinking and introduce the tools pertinent to its pursuit. The tutorial is designed to be self-contained and will cover concepts from introductory to advanced. Both practitioners and researchers with an interest in the impact of value considerations in software decision-making will benefit from attending it.This tutorial is offered in conjunction with the Fourth International Workshop on Economics-Driven Software Engineering Research (EDSER-4). The tutorial is meant in part to enable those who would like to participate in the workshop, but who might not possess the requisite background, to come up to speed.

CASE (computer aided software engineering) tools are believed to have played a critical role in improving software productivity and quality by assisting tasks in software development processes since the 1970s. Several parametric software cost models adopt "use of software tools" as one of the environmental factors that affects software development productivity. Several software cost models assess the productivity impacts of CASE tools based only on breadth of tool coverage without considering other productivity dimensions such as degree of integration, tool maturity, and user support. This paper provides an extended set of tool rating scales based on the completeness of tool coverage, the degree of tool integration, and tool maturity/user support. Those scales are used to refine the way in which CASE tools are effectively evaluated within COCOMO (constructive cost model) II. In order to find the best fit of weighting values for the extended set of tool rating scales in the extended research model, a Bayesian approach is adopted to combine two sources of (expert-judged and data-determined) information to increase prediction accuracy. The extended model using the three TOOL rating scales is validated by using the cross-validation methodologies, data splitting, and bootstrapping. This approach can be used to disaggregate other parameters that have significant impacts on software development productivity and to calibrate the best-fit weight values based on data-determined and expert-judged distributions. It results in an increase in the prediction accuracy in software parametric cost estimation models and an improvement in insights on software productivity investments.

The University of Southern California (USC) teaches a two-semester real-client project course as a core course in USC's MSCS-SE (Master of Science in Computer Science and Software Engineering) degree program. The course has evolved rapidly, each year introducing many changes in order to satisfy the course stakeholder's win conditions. The course has also been our primary experimental testbed for evolving our MBASE (Model-Based Architecting and Software Engineering) model integration framework. In turn, this framework, along with the CRESST (Center for Research on Evaluation, Standards and Student Testing) cognitive demands analysis has served as an effective means of managing the course's rapid evolution. A further test of the framework has been its application to undergraduate software engineering project courses at other institutions. This paper provides a description and examples of USC's experience in constructing and evolving a family of software engineering project courses based on the MBASE software engineering model integration framework. A discussion of course needs, use of software engineering models, a description of MBASE, use of the CRESST model of learning objectives, course development and experiences are presented. The approach has been used successfully at Columbia University and to a lesser degree at other institutions. Much of the current and historical materials and resources described in this paper are freely available for educational use.

In 1996, the University of Southern California (USC) switched its core two-semester software engineering course from a hypothetical-project, homework-and-exam course based on the Bloom taxonomy of educational objectives (knowledge, comprehension, application, analysis, synthesis and evaluation). The revised course is a real-client team-project course based on the CRESST (Center for Research on Evaluation, Standards and Student Testing) model of learning objectives (content understanding, problem solving, collaboration, communication and self-regulation). We used the CRESST cognitive demands analysis to determine the necessary student skills required for software risk management and the other major project activities, and have been refining the approach over the last four years of experience, including revised versions for one-semester undergraduate and graduate project courses at Columbia University. This paper summarizes our experiences in evolving the risk management aspects of the project courses. These have helped us mature more general techniques, such as risk-driven specifications, domain-specific simplifier and complicator lists, and the SAIV (schedule as an independent variable) process model. The largely positive results in terms of review pass/fail rates, client evaluations, product adoption rates and hiring manager feedback are summarized as well.

Describes the application of the WinWin paradigm to identify and resolve conflicts in a series of real-client, student-developer digital library projects. The paper is based on a case study of the statistical analysis of 15 projects and an in-depth analysis of one representative project. These analyses focus on the conflict resolution process, stakeholders' roles and their relationships to quality artifacts, and tool effectiveness. We show that stakeholders tend to accept satisfactory rather than optimal resolutions. Users and customers are more proactive in stating win conditions, whereas developers are more active in working toward resolutions. Further, we suggest that knowledge-based automated aids have potential to significantly enhance process effectiveness and efficiency. Finally, we conclude that such processes and tools have theoretical and practical implications in the quest for better software requirements elicitation.

EasyWinWin is a requirements definitions methodology that builds on the win-win negotiation approach and leverages collaborative technology to improve the involvement and interaction of key stakeholders. With EasyWinWin, stakeholders move through a step-by-step win-win negotiation where they collect, elaborate, and prioritize their requirements, and then surface and resolve issues.

Since 1996 the authors have had great success combining software engineering research and software engineering education. This effort introduces new research and experimental areas into the curriculum along with the means to evolve and refine current practices. The curriculum itself becomes a valuable research area with direct applications to real-world practice. The approach in particular tries to deal with four fundamental issues within software engineering and software engineering education. There are Exposing students to a no-surprises full lifecycle software engineering experience. Dealing with the increasing distribution and diversity of students. Providing students a non-trivial real-world customer-based development experience within the constraints of the university. Introducing software engineering research and industry best practices into the curriculum.

Software vendors rarely acknowledge their debt to research, indeed often are unaware of it, and rarely even appreciate the importance of such acknowledgement. The long lead times, and tortuous adoption paths, for software engineering research contributions also cloud perception of the actual source of popularly adopted software engineering technologies.

One of the difficulties in requirements negotiation is to determine a feasible and mutually satisfactory set of requirements for the developer and the user, a problem related to C.P. Snow's (1959) "Two Cultures" problem. During the last year of our experience with an annual series of digital library projects, we have been experimenting with expectations management and domain specific lists of "simplifiers" and "complicators", as a way to address the "Two Cultures" problem involving librarians and computer scientists. Initial results indicate that the simplifiers and complicators approach successfully reduced the number of projects having serious feasibility problems, and helped manage the expectations of both the developers and the customers/users. We see no obstacles to applying the approach to other domains.

Many parametric software estimation models have evolved in the last two decades (L.H. Putnam and W. Myers, 1992; C. Jones, 1997; R.M. Park et al., 1992). Almost all of these parametric models have been empirically calibrated to actual data from completed software projects. The most commonly used technique for empirical calibration has been the popular classical multiple regression approach. As discussed in the paper, the multiple regression approach imposes a few assumptions frequently violated by software engineering datasets. The paper illustrates the problems faced by the multiple regression approach during the calibration of one of the popular software engineering cost models, COCOMO II. It describes the use of a pragmatic 10 percent weighted average approach that was used for the first publicly available calibrated version (S. Chulani et al., 1998). It then moves on to show how a more sophisticated Bayesian approach can be used to alleviate some of the problems faced by multiple regression. It compares and contrasts the two empirical approaches, and concludes that the Bayesian approach was better and more robust than the multiple regression approach.

Negotiating requirements is one of the first steps in any software system life cycle, but its results have probably the most significant impact on the system's value. However, the processes of requirements negotiation are not well understood. We have had the opportunity to capture and analyze requirements negotiation behavior for groups of projects developing library multimedia archive systems, using an instrumented version of the USC WinWin groupware system for requirements negotiation. Some of the more illuminating results were: most stakeholder Win Conditions were noncontroversial (were not involved in issues); negotiation activity varied by stakeholder role; LCO package quality (measured by grading criteria) could be predicted by negotiation attributes; and WinWin increased cooperativeness, reduced friction, and helped focus on key issues.

The COCOMO II model was created to meet the need for a cost model that accounted for future software development practices. This paper describes some of the experiences learned in calibrating COCOMO II Post-Architecture model from eighty-three observations. The results of the multiple regression analysis, their implications, and a future calibration strategy are discussed.

The process of resolving cost-requirements conflicts is difficult because of incompatibilities among stakeholders' interests and priorities, complex cost requirements dependencies, and an exponentially increasing option space for larger systems. The paper describes an exploratory knowledge-based tool,the Software Cost Option Strategy Tool (S-COST), for assisting stakeholders to surface appropriate cost resolution options, to visualize the options, and to negotiate a mutually satisfactory balance of requirements and cost. S-COST operates in the context of the USC-CSE WinWin system (a groupware support system for determining software and system requirements as negotiated win conditions), QARCC (a support system for identifying conflicts in quality requirements), and COCOMO (constructive cost estimation model). Initial analyses of its capabilities indicate that its semiautomated approach provides users with improved capabilities for addressing cost-requirements issues.

No abstract available.

A major problem in requirements engineering is obtaining requirements that address the concerns of multiplestakeholders.An approach to such a problem is the Theory-W based Spiral Model.This paper focuses on the problem of developing a support system for such a model.In particular

Abstract-A good software project management theory should he simultaneously simple, general, and specific. To date, those objectives have been difficult to satisfy. This paper presents a candidate software management theory and shows that it satisfies those objectives reason-ably well. Reflecting various alphabetical management theories (X, Y, Z), it is called the Theory W approach to software project manage-ment.

A discussion is presented of the two primary ways of understanding software costs. The black-box or influence-function approach provides useful experimental and observational insights on the relative software productivity and quality leverage of various management, technical, environmental, and personnel options. The glass-box or cost distribution approach helps identify strategies for integrated software productivity and quality improvement programs using such structures as the value chain and the software productivity opportunity tree. The individual strategies for improving software productivity are identified. Issues related to software costs and controlling them are examined and discussed. It is pointed out that a good framework of techniques exists for controlling software budgets, schedules, and work completed, but that a great deal of further progress is needed to provide an overall set of planning and control techniques covering software product qualities and end-user system objectives.>

In this experiment, seven software teams developed versions of the same small-size (2000-4000 source instruction) application software product. Four teams used the Specifying approach. Three teams used the Prototyping approach. The main results of the experiment were the following. 1) Prototyping yielded products with roughly equivalent performance, but with about 40 percent less code and 45 percent less effort. 2) The prototyped products rated somewhat lower on functionality and robustness, but higher on ease of use and ease of learning. 3) Specifying produced more coherent designs and software that was easier to integrate. The paper presents the experimental data supporting these and a number of additional conclusions.

This paper reports the results of an experiment in applying large-scale software engineering procedures to small software projects. Two USC student teams developed a small, interactive application software product to the same specification, one using Fortran and one using Pascal. Several hypotheses were tested, and extensive experimenal data collected. The major conclusions were as follows.

This paper provides a definition of the term "software engineering" and a survey of the current state of the art and likely future trends in the field. The survey covers the technology available in the various phases of the software life cycle—requirements engineering, design, coding, test, and maintenance—and in the overall area of software management and integrated technology-management approaches. It is oriented primarily toward discussing the domain of applicability of techniques (where and when they work), rather than how they work in detail. To cover the latter, an extensive set of 104 references is provided.

Summarizes some recent experience in analyzing and eliminating sources of error in the design phase of large software projects. It points out some of the significant differences in software error incidence between large and small software projects. A taxonomy of software error causes, and some analyses of the design error data performed to obtain a better understanding of the nature of large-scale software design errors and to evaluate alternative methods of preventing, detecting, and eliminating them are presented. Based on this analysis of observational data, a hypothesis was derived regarding the potential cost effectiveness of an automated aid to detecting inconsistencies between assertions about the nature of inputs and outputs of the various elements of the software design.

This paper describes an operational interactive computer graphics system, called Graphic ROCKET, currently being used as a performance analysis aid for preliminary design and evaluation of space vehicle systems. It provides means for rapidly specifying and evaluating the performance of a wide range aerospace vehicle designs and flight plans.

POGO is an operational interactive graphics system which allows one to create inter- active graphics programs with relatively little elapsed time, programming expertise, or graphics expertise. It features a design program for composing CRT control “pages” by example instead of by programming, and facilities for cleanly inter- facing such pages with FORTRAN computational and control subroutines, and with a standard set of curve input and display pages. The presentation will include a description of our development and usage experience, and a film showing the use of POGO in creating interactive graphics programs for trajectory analysis, for analysis of fluid balance in the human body, and for creating parts of POGO itself.