Security threats should be identified in the early phases of a project so that design solutions can be explored and mitigating requirements specified. In this paper, we present a crowd-sourcing approach for creating Personae non Gratae (PnGs), which model attack goals and techniques of unwanted, potentially malicious users. We present a proof of concept study that takes a diverse collection of potentially redundant PnGs and merges them into a single set. Our approach combines machine learning techniques and visualization. It is illustrated and evaluated using a collection of PnGs collected from undergraduate students for a drone-based rescue scenario. Lessons learned from the proof of concept study are discussed and lay the foundations for future work.

Programming languages can restrict state change by preventing it entirely (immutability) or by restricting which clients may modify state (read-only restrictions). The benefits of immutability and read-only restrictions in software structures have been long-argued by practicing software engineers, researchers, and programming language designers. However, there are many proposals for language mechanisms for restricting state change, with a remarkable diversity of techniques and goals, and there is little empirical data regarding what practicing software engineers want in their tools and what would benefit them. We systematized the large collection of techniques used by programming languages to help programmers prevent undesired changes in state. We interviewed expert software engineers to discover their expectations and requirements, and found that important requirements, such as expressing immutability constraints, were not reflected in features available in the languages participants used. The interview results informed our design of a new language extension for specifying immutability in Java. Through an iterative, participatory design process, we created a tool that reflects requirements from both our interviews and the research literature.

Presents an editorial discusses the current status and activities supported by this publication.

Differing perceptions and expectations are obstaclesto collaboration between software engineering (SE) researchersand practitioners: Researchers often have a view thatpractitioners are reluctant to share real data. Practitionersbelieve that researchers are mostly working on topics which aredivorced from real industrial needs. Researchers believe thatpractitioners are looking for quick fixes. Practitioners have aview that case studies in research do not represent thecomplexities of real projects. Researchers may expect a few yearsto do research on a problem whereas practitioners expect a quicksolution that pays off immediately.Researchers and practitioners need to identify the gaps and todiscover the ways to collaborate to strengthen SE research andindustrial practice (IP). The main purpose of this workshop is tobring together researchers and practitioners to discuss thecurrent state of SE research and IP and to enhance collaborationbetween them. The SER&IP 2015 workshop provided a platformto share success stories of SE research-practice partnerships aswell as to discuss the challenges, through a day-long agenda ofkeynotes, paper presentations and round table discussions.

Context: The technical debt (TD) concept describes a tradeoff between short-term and long-term goals in software development. While it is highly useful as a metaphor, it has utility beyond the facilitation of discussion, to inspire a useful set of methods and tools that support the identification, measurement, monitoring, management, and payment of TD. Objective: This study focuses on the identification of TD. We evaluate human elicitation of TD and compare it to automated identification. Method: We asked a development team to identify TD items in artifacts from a software project on which they were working. We provided the participants with a TD template and a short questionnaire. In addition, we also collected the output of three tools to automatically identify TD and compared it to the results of human elicitation. Results: There is little overlap between the TD reported by different developers, so aggregation, rather than consensus, is an appropriate way to combine TD reported by multiple developers. The tools used are especially useful for identifying defect debt but cannot help in identifying many other types of debt, so involving humans in the identification process is necessary. Conclusion: We have conducted a case study that focuses on the practical identification of TD, one area that could be facilitated by tools and techniques. It contributes to the TD landscape, which depicts an understanding of relationships between different types of debt and how they are best discovered.

Existing research is unclear on how to generate lessons learned for defect prediction and effort estimation. Should we seek lessons that are global to multiple projects or just local to particular projects? This paper aims to comparatively evaluate local versus global lessons learned for effort estimation and defect prediction. We applied automated clustering tools to effort and defect datasets from the PROMISE repository. Rule learners generated lessons learned from all the data, from local projects, or just from each cluster. The results indicate that the lessons learned after combining small parts of different data sources (i.e., the clusters) were superior to either generalizations formed over all the data or local lessons formed from particular projects. We conclude that when researchers attempt to draw lessons from some historical data source, they should 1) ignore any existing local divisions into multiple sources, 2) cluster across all available data, then 3) restrict the learning of lessons to the clusters from other sources that are nearest to the test data.

Background: Existing software defect classification schemes support multiple tasks, such as root cause analysis and process improvement guidance. However, existing schemes do not assist in assigning defects to a broad range of high level software goals, such as software quality characteristics like functionality, maintainability, and usability. Aim: We investigate whether a classification based on the ISO/IEC 9126 software product quality model is reliable and useful to link defects to quality aspects impacted. Method: Six different subjects, divided in two groups with respect to their expertise, classified 78 defects from an industrial web application using the ISO/IEC 9126 quality main characteristics and sub-characteristics, and a set of proposed extended guidelines. Results: The ISO/IEC 9126 model is reasonably reliable when used to classify defects, even using incomplete defect reports. Reliability and variability is better for the six high level main characteristics of the model than for the 22 subcharacteristics. Conclusions: The ISO/IEC 9126 software quality model provides a solid foundation for defect classification. We also recommend, based on the follow up qualitative analysis performed, to use more complete defect reports and tailor the quality model to the context of use.

A significant body of knowledge concerning software inspection practice indicates that the value of inspections varies widely both within and across organizations. Inspection effectiveness and efficiency may be affected by a variety of factors such as inspection planning, the type of software, the developing organization, and many others. In the early 1990's, a governmental organization developing complex and highly critical software systems formulated heuristics for inspection planning based on best practices and their early inspection data. Since the development context at the organization has changed in some ways since the heuristics were proposed, it is important to assess whether the heuristics are still a suitable guideline to use. To investigate this question, we statistically evaluated the differences in effectiveness and efficiency between inspections that adhered to the heuristics and ones that did not. Our analysis revealed no significant difference in effectiveness or efficiency for most heuristics. We also learned that compliance with the heuristics is diminishing over time.

Background: Automatic static analysis (ASA) tools examine source code to discover “issues”, i.e. code patterns that are symptoms of bad programming practices and that can lead to defective behavior. Studies in the literature have shown that these tools find defects earlier than other verification activities, but they produce a substantial number of false positive warnings. For this reason, an alternative approach is to use the set of ASA issues to identify defect prone files and components rather than focusing on the individual issues. Aim: We conducted an exploratory study to investigate whether ASA issues can be used as early indicators of faulty files and components and, for the first time, whether they point to a decay of specific software quality attributes, such as maintainability or functionality. Our aim is to understand the critical parameters and feasibility of such an approach to feed into future research on more specific quality and defect prediction models. Method: We analyzed an industrial C# web application using the Resharper ASA tool and explored if significant correlations exist in such a data set. Results: We found promising results when predicting defectprone files. A set of specific Resharper categories are better indicators of faulty files than common software metrics or the collection of issues of all issue categories, and these categories correlate to different software quality attributes. Conclusions: Our advice for future research is to perform analysis on file rather component level and to evaluate the generalizability of categories. We also recommend using larger datasets as we learned that data sparseness can lead to challenges in the proposed analysis process.

Identifying refactoring opportunities in software systems is an important activity in today's agile development environments. The concept of code smells has been proposed to characterize different types of design shortcomings in code. Additionally, metric-based detection algorithms claim to identify the "smelly" components automatically. This paper presents results for an empirical study performed in a commercial environment. The study investigates the way professional software developers detect god class code smells, then compares these results to automatic classification. The results show that, even though the subjects perceive detecting god classes as an easy task, the agreement for the classification is low. Misplaced methods are a strong driver for letting subjects identify god classes as such. Earlier proposed metric-based detection approaches performed well compared to the human classification. These results lead to the conclusion that an automated metric-based pre-selection decreases the effort spent on manual code inspections. Automatic detection accompanied by a manual review increases the overall confidence in the results of metric-based classifiers.

Adapting new software processes and practices in organizational and academic environments requires training the developers and validating the applicability of the newly introduced activities. Investigating process conformance during training and understanding if programmers are able and willing to follow the specific steps are crucial to evaluating whether the process improves various software product quality factors. In this paper we present a process model independent approach to detect process non-conformance. Our approach is based on non-intrusively collected data captured by a version control system and provides the project manager with timely updates. Further, we provide evidence of the applicability of our approach by investigating process conformance in a five day training class on eXtreme Programming (XP) practices at the Leibniz Universität Hannover. Our results show that the approach enabled researchers to formulate minimal intrusive methods to check for conformance and that for the majority of the investigated XP practices violations could be detected.

Code smells refer to commonly occurring patterns in source code that indicate poor programming practices or code decay. Detecting code smells helps developers find design problems that can cause trouble in future maintenance. Detection rules for code smells, based on software metrics, have been proposed, but they do not take domain-specific characteristics into consideration. In this study we investigate whether such generic heuristics can be tailored to include domain-specific factors. Input into these domain-specific heuristics comes from an iterative empirical field study in a software maintenance project. The results yield valuable insight into code smell detection.

Abstract This paper reports on an industrial case study in a large Norwegian Oil and Gas company (StatoilHydro ASA) involving a reusable Java‐class framework and two applications that use that framework. We analyzed software changes from three releases of the reusable framework, called Java Enterprise Framework (JEF), and two applications reusing the framework, called Digital Cargo File (DCF) and Shipment and Allocation (S&A). On the basis of our analysis, we found the following: (1) Profiles of change types for the reused framework and the applications are similar, specifically, perfective changes dominate significantly. (2) Although on observing the mean value adaptive changes are more frequent and are active longer in JEF and S&A, these systems went through less refactoring than DCF. For DCF, we saw that preventive changes were more frequent and were active longer. (3) Finally, we found that designing for reuse seems to lead to a long‐term payoff in relation to non‐reusable software systems. Copyright © 2010 John Wiley & Sons, Ltd.

In the past decades the software engineering community has proposed a large collection of software development life cycles, models, and processes. The goal of a major set of these processes is to assure that the product is finished within time and budget, and that a predefined set of functional and nonfunctional requirements (e.g. quality goals) are satisfied at delivery time. Based upon the assumption that there is a real relationship between the process applied and the characteristics of the product developed from that process, we developed a tool supported approach that uses process nonconformance detection to identify potential risks in achieving the required process characteristics. In this paper we present the approach and a feasibility study that demonstrates its use on a large-scale software development project in the aerospace domain. We demonstrate that our approach, in addition to meeting the criteria above, can be applied to a real system of reasonable size; can represent a useful and adequate set of rules of relevance in such an environment; and can detect relevant examples of process nonconformance that provide useful insight to the project manager.

This paper describes our experience in aggregating a number of historical datasets containing inspection defect data using different categorization schemes. Our goal was to make use of the historical data by creating models to guide future development projects. We describe our approach to reconciling the different choices used in the historical datasets to categorize defects, and the challenges we faced. We also present a set of recommendations for others involved in classifying defects.

Experimental replications are very important to the advancement of empirical software engineering. Replications are one of the key mechanisms to confirm previous experimental findings. They are also used to transfer experimental knowledge, to train people, and to expand a base of experimental evidence. Unfortunately, experimental replications are difficult endeavors. It is not easy to transfer experimental know-how and experimental findings. Based on our experience, this paper discusses this problem and proposes a Framework for Improving the Replication of Experiments (FIRE). The FIRE addresses knowledge sharing issues both at the intra-group (internal replications) and inter-group (external replications) levels. It encourages coordination of replications in order to facilitate knowledge transfer for lower cost, higher quality replications and more generalizable results.

There is widespread belief in the computer science community that MPI is a difficult and time-intensive approach to developing parallel software. Nevertheless, MPI remains the dominant programming model for HPC systems, and many projects have made effective use of it. It remains unknown how much impact the use of MPI truly has on the productivity of computational scientists. In this paper, we examine a mature, ongoing HPC project, the Flash Center at the University of Chicago, to understand how MPI is used and to estimate the time that programmers spend on MPI-related issues during development. Our analysis is based on an examination of the source code, version control history, and regression testing history of the software. Based on our study, we estimate that about 20% of the development effort is related to MPI. This implies a maximum productivity improvement of 25% for switching to an alternate parallel programming model.

Inspections are widely used and studies have found them to be effective in uncovering defects. However, there is less data available regarding the impact of inspections on different defect types and almost no data quantifying the link between inspections and desired end product qualities. This paper addresses this issue by investigating whether design inspection checklists can be tailored so as to effectively target certain defect types without impairing the overall defect detection rate. The results show that the design inspection approach used here does uncover useful design quality issues and that the checklists can be effectively tailored for some types of defects.

Providing robust decision support for software engineering (SE) requires the collection of data across multiple contexts so that one can begin to elicit the context variables that can influence the results of applying a technology. However, the task of comparing contexts is complex due to the large number of variables involved. This works extends a previous one in which we proposed a practical and rigorous process for identifying evidence and context information from SE papers. The current work proposes a specific template to collect context information from SE papers and an interactive approach to compare context information about these studies. It uses visualization and clustering algorithms to help the exploration of similarities and differences among empirical studies. This paper presents this approach and a feasibility study in which the approach is applied to cluster a set of papers that were independently grouped by experts.

The number of scientific publications is constantly increasing, and the results published on Empirical Software Engineering are growing even faster. Some software engineering publishers have begun to collaborate with research groups to make available repositories of software engineering empirical data. However, these initiatives are limited due to data ownership and privacy issues. As a result, many researchers in the area have adopted systematic reviews as a mean to extract empirical evidence from published material. Systematic reviews are labor intensive and costly. In this paper, we argue that the use of Information Extraction Tools can support systematic reviews and significantly speed up the creation of repositories of SE empirical evidence.

The medical device industry is a constantly growing domain which makes use of more and more software products. Given the importance to this industry of dependable software components, rigorous software engineering techniques would seem to have an important role to play. However, in a recent survey of the industry we found a lower than expected rate of adoption of certain sound software engineering practices. To ensure and improve the quality of developed software products, whether they are standalone applications or embedded in complex systems, both the medical device industry as well as software engineers have to take action. Our survey is a first step in this direction and may help in identifying appropriate programs and future research topics.

Selecting the right practice or technology for a given task should be based on the project goals and context. The selection process should be supported by empirical evidence and experiences with the practice in similar contexts [1]. Practitioners, however, usually do not have the time to search for existing empirical studies and to evaluate them in detail. A more centralized and practice-oriented approach is necessary to provide the required information in a convenient way. EMPEROR (experience management portal using empirical results as organizational resources) was developed to address this issue.

In developing High-Performance Computing (HPC) software, time to solution is an important metric. This metric is comprised of two main components: the human effort required developing the software, plus the amount of machine time required to execute it. To date, little empirical work has been done to study the first component: the human effort required and the effects of approaches and practices that may be used to reduce it. In this paper, we describe a series of studies that address this problem. We instrumented the development process used in multiple HPC classroom environments. We analyzed data within and across such studies, varying factors such as the parallel programming model used and the application being developed, to understand their impact on the development process.

In support of decision-making for planning the effort to be allocated to inspections in different software development phases, we propose combining empirical studies with process modeling and simulation. We present the simulator developed for answering questions and running "what-if" scenarios specific to NASA software development projects.

Selecting technologies for developing software is a crucial activity in software projects. Defect reduction is an example of an area in which software developers have to decide what technologies to use. CeBASE is a NSF funded project that has the role of improving software development by providing decision support on the selection of techniques and tools. The decision support is based on empirical data organized in experience bases and refined into high-level models. Empirical data is collected through various activities, for example through eWorkshops in which experts discuss important issues, and formalized using the lightweight knowledge dust to knowledge pearl process.

No abstract available.

There is a growing interest in empirical study in software engineering, both for validating mature technologies and for guiding improvements of less-mature technologies. This paper introduces an empirical methodology, based on experiences garnered over more than two decades of work by the Empirical Software Engineering Group at the University of Maryland and related organizations, for taking a newly proposed improvement to development processes from the conceptual phase through transfer to industry. The methodology presents a series of questions that should be addressed, as well as the types of studies that best address those questions. The methodology is illustrated by a specific research program on inspection processes for Object-Oriented designs. Specific examples of the studies that were performed and how the methodology impacted the development of the inspection process are also described.

No abstract available.

The empirical study described in the paper addresses software reading for construction: how application developers obtain an understanding of a software artifact for use in new system development. The study focuses on the processes that developers would engage in when learning and using object oriented frameworks. We analyzed 15 student software development projects using both qualitative and quantitative methods to gain insight into what processes occurred during framework usage. The contribution of the study is not to test predefined hypotheses but to generate well-supported hypotheses for further investigation. The main hypotheses produced are that example based techniques are well suited to use by beginning learners, while hierarchy based techniques are not, because of a larger learning curve. Other more specific hypotheses are proposed and discussed.

Inspections can be used to identify defects in software artifacts. In this way, inspection methods help to improve software quality, especially when used early in software development. Inspections of software design may be especially crucial since design defects (problems of correctness and completeness with respect to the requirements, internal consistency, or other quality attributes) can directly affect the quality of, and effort required for, the implementation.

Experimentation in software engineering is necessary but difficult. One reason is that there are a large number of context variables and, so, creating a cohesive understanding of experimental results requires a mechanism for motivating studies and integrating results. It requires a community of researchers that can replicate studies, vary context variables, and build models that represent the common observations about the discipline. The paper discusses the experience of the authors, based upon a collection of experiments, in terms of a framework for organizing sets of related studies. With such a framework, experiments can be viewed as part of common families of studies, rather than being isolated events. Common families of studies can contribute to important and relevant hypotheses that may not be suggested by individual experiments. A framework also facilitates building knowledge in an incremental manner through the replication of experiments within families of studies. To support the framework, the paper discusses the experiences of the authors in carrying out empirical studies, with specific emphasis on persistent problems encountered in experimental design, threats to validity, criteria for evaluation, and execution of experiments in the domain of software engineering.

The empirical study described in this paper focuses on the effectiveness of maintenance processes in an environment in which a repository of potential sources of reuse exists, e.g. a context in which applications are built using an object-oriented framework. Such a repository might contain current and previous releases of the system under maintenance, as well as other applications that are built on a similar structure or contain similar functionality. This paper presents an observational study of 15 student projects in framework-based environment. We used a mix of qualitative and quantitative methods to identify and evaluate the effectiveness of the maintenance processes.