The term and label, “case study”, is not used consistently by authors of primary studies in software engineering research. It is not clear whether this problem also occurs for systematic literature studies (SLSs). To investigate the extent to which SLSs in/correctly use the term and label, “case study”, when classifying primary studies. We systematically collect two sub-samples (2010–2021 & 2022) comprising a total of eleven SLSs and 79 primary studies. We examine the designs of these SLSs, and then analyse whether the SLS authors and the primary-study authors correctly label the respective primary study as a “case study”. 76% of the 79 primary studies are misclassified by SLSs (with the two sub-samples having 60% and 81% misclassification, respectively). For 39% of the 79 studies, the SLSs propagate a mislabelling by the original authors, whilst for 37%, the SLSs introduce a new mislabel, thus making the problem worse. SLSs rarely present explicit definitions for “case study” and when they do, the definition is not consistent with established definitions. SLSs are both propagating and exacerbating the problem of the mislabelling of primary studies as “case studies”, rather than – as we should expect of SLSs – correcting the labelling of primary studies, and thus improving the body of credible evidence. Propagating and exacerbating mislabelling undermines the credibility of evidence in terms of its quantity, quality and relevance to both practice and research.

None

Abstract During software evolution, decisions related to components' origin or source significantly impact the quality properties of the product and development metrics such as cost, time to market, ease of maintenance, and further evolution. Thus, such decisions should ideally be supported by evidence, i.e., using previous experiences and information from different sources, even own previous experiences. A hindering factor to such reuse of previous experiences is that these decisions are highly context‐dependent and it is difficult to identify when previous experiences come from sufficiently similar contexts to be useful in a current setting. Conversely, when documenting a decision (as a decision experience), it is difficult to know which context factors will be most beneficial when reusing the experience in the future. An interview study is performed to identify a list of context factors that are perceived to be most important by practitioners when using experiences to support decision‐making for component sourcing, using a specific scenario with alternative sources of experiences. We observed that the further away (from a company or an interviewee) the experience evidence is, as is the case for online experiences, the more context factors are perceived as important by practitioners to make use of the experience. Furthermore, we discuss and identify further research to make this type of decision‐making more evidence‐based.

Software engineering has moved towards being a more evidence-based discipline. For example, systematic literature studies have become a standard practice in our field. However, to be evidence-based, we, as authors of both primary and systematic literature studies, still need to ensure that we publish credible evidence that researchers and practitioners alike can build upon. The keynote will discuss evidence and illustrate that we need to take action to become more evidence-based and that we are all responsible for improving as authors, reviewers, program chairs and editors. Some of the problems are illustrated generally, and then the findings from analyzing a sample of requirements engineering papers are presented before concluding the presentation.

The definition and term “case study” are not being applied consistently by software engineering researchers. We previously developed a trivial “smell indicator” to help detect the misclassification of primary studies as case studies. To evaluate the performance of the indicator. We compare the performance of the indicator against human classifiers for three datasets, two datasets comprising classifications by both authors of systematic literature studies and primary studies, and one dataset comprising only primary-study author classifications. The indicator outperforms the human classifiers for all datasets. The indicator is successful because human classifiers “fail” to properly classify their own, and others’, primary studies. Consequently, reviewers of primary studies and authors of systematic literature studies could use the classifier as a “sanity” check for primary studies. Moreover, authors might use the indicator to double-check how they classified a study, as part of their analysis, and prior to submitting their manuscript for publication. We challenge the research community to both beat the indicator, and to improve its ability to identify true case studies.

Software practitioners are a primary provider of information for field studies in software engineering. Research typically recruits practitioners through some kind of sampling. But sampling may not in itself recruit the “right” participants. To assess existing guidance on participant recruitment, and to propose and illustrate a framework for recruiting professional practitioners as credible participants in field studies of software engineering. We review existing guidelines, checklists and other advisory sources on recruiting participants for field studies. We develop a framework, partly based on our prior research and on the research of others. We search for and select three exemplar studies (a case study, an interview study and a survey study) and use those to illustrate the framework. Whilst existing guidance recognises the importance of recruiting participants, there is limited guidance on how to recruit the “right” participants. The framework suggests the conceptualisation of participants as “research instruments” or, alternatively, as a sampling frame for items of interest. The exemplars suggest that at least some members of the research community are aware of the need to carefully recruit the “right” participants. The framework is intended to encourage researchers to think differently about the involvement of practitioners in field studies of software engineering. Also, the framework identifies a number of characteristics not explicitly addressed by existing guidelines.

A good search strategy is essential for a successful systematic literature study. Historically, database searches have been the norm, which was later complemented with snowball searches. Our conjecture is that we can perform even better searches if combining these two search approaches, referred to as a hybrid search strategy. Our main objective was to compare and evaluate a hybrid search strategy. Furthermore, we compared four alternative hybrid search strategies to assess whether we could identify more cost-efficient ways of searching for relevant primary studies. To compare and evaluate the hybrid search strategy, we replicated the search procedure in a systematic literature review (SLR) on industry–academia collaboration in software engineering. The SLR used a more “traditional” approach to searching for relevant articles for an SLR, while our replication was executed using a hybrid search strategy. In our evaluation, the hybrid search strategy was superior in identifying relevant primary studies. It identified 30% more primary studies and even more studies when focusing only on peer-reviewed articles. To embrace individual viewpoints when assessing research articles and minimise the risk of missing primary studies, we introduced two new concepts, wild cards and borderline articles, when performing systematic literature studies. The hybrid search strategy is a strong contender for being used when performing systematic literature studies. Furthermore, alternative hybrid search strategies may be viable if selected wisely in relation to the start set for snowballing. Finally, the two new concepts were judged as essential to cater for different individual judgements and to minimise the risk of excluding primary studies that ought to be included.

The term “case study” is not used consistently when describing studies and, most importantly, is not used according to the established definitions. Given the misuse of the term “case study”, we critically analyse articles that cite case study guidelines and report case studies. We find that only about 50% of the studies labelled “case study” are correctly labelled, and about 40% of studies labelled “case study” are actually better understood as “small-scale evaluations”. Based on our experiences conducting the analysis, we formulate support for ensuring and assuring the correct labelling of case studies. We develop a checklist and a self-assessment scheme. The checklist is intended to complement existing definitions and to encourage researchers to use the term “case study” correctly. The self-assessment scheme is intended to help the researcher identify when their empirical study is a “small-scale evaluation” and, again, encourages researchers to label their studies correctly. Finally, we develop and evaluate a smell indicator to automatically suggest when a reported case study may not actually be a case study. These three instruments have been developed to help ensure and assure that only those studies that are actually case studies are labelled as “case study”.

Case studies are regularly published in the software engineering literature, and guidelines for conducting case studies are available. Based on a perception that the label “case study” is assigned to studies that are not case studies, an investigation has been conducted. The aim was to investigate whether or not the label “case study” is correctly used in software engineering research. To address the objective, 100 recent articles found through Scopus when searching for case studies in software engineering have been investigated and classified. Unfortunately, the perception of misuse of the label “case study” is correct. Close to 50% of the articles investigated were judged as not being case studies according to the definition of a case study. We either need to ensure correct use of the label “case study”, or we need another label for its definition. Given that “case study” is a well-established label, it is probably impossible to change the label. Thus, we introduce an alternative definition of case study emphasising its real-life context, and urge researchers to carefully follow the definition of different research methods when presenting their research.

Context: An evidence-based scientific discipline should produce, consume and disseminate credible evidence. Unfortunately, mistakes are sometimes made, resulting in the production, consumption and dissemination of invalid or otherwise questionable evidence. In the worst cases, such questionable evidence achieves the status of accepted knowledge. There is, therefore, the need to ensure that producers and consumers seek to identify and rectify such situations. Objectives: To raise awareness of the negative impact of misinterpreting evidence and of propagating that misinterpreted evidence, and to provide guidance on how to improve on the type of issues identified. Methods: We use a case-based approach to present and analyse the production, consumption and dissemination of evidence. The cases are based on the literature and our professional experience. These cases illustrate a range of challenges confronting evidence-based researchers as well as the consequences to research when invalid evidence is not corrected in a timely way. Results: We use the cases and the challenges to formulate a framework and a set of recommendations to help the community in producing and consuming credible evidence. Conclusions: We encourage the community to collectively remain alert to the emergence and dissemination of invalid, or otherwise questionable, evidence, and to proactively seek to identify and rectify it.

The literature concerning research methodologies and methods has increased in software engineering in the last decade. However, there is limited guidance on selecting an appropriate research methodology for a given research study or project. Based on a selection of research methodologies suitable for software engineering research in collaboration between industry and academia, we present, discuss and compare the methodologies aiming to provide guidance on which research methodology to choose in a given situation to ensure successful industry–academia collaboration in research. Three research methodologies were chosen for two main reasons. Design Science and Action Research were selected for their usage in software engineering. We also chose a model emanating from software engineering, i.e., the Technology Transfer Model. An overview of each methodology is provided. It is followed by a discussion and an illustration concerning their use in industry–academia collaborative research. The three methodologies are then compared using a set of criteria as a basis for our guidance. The discussion and comparison of the three research methodologies revealed general similarities and distinct differences. All three research methodologies are easily mapped to the general research process describe–solve–practice, while the main driver behind the formulation of the research methodologies is different. Thus, we guide in selecting a research methodology given the primary research objective for a given research study or project in collaboration between industry and academia. We observe that the three research methodologies have different main objectives and differ in some characteristics, although still having a lot in common. We conclude that it is vital to make an informed decision concerning which research methodology to use. The presentation and comparison aim to guide selecting an appropriate research methodology when conducting research in collaboration between industry and academia.

Abstract Software engineering is decision intensive. Evidence‐based software engineering is suggested for decision‐making concerning the use of methods and technologies when developing software. Software development often includes the reuse of software assets, for example, open‐source components. Which components to use have implications on the quality of the software (e.g., maintainability). Thus, research is needed to support decision‐making for composite software. This paper presents a roadmap for research required to support evidence‐based decision‐making for choosing and integrating assets in composite software systems. The roadmap is developed as an output from a 5‐year project in the area, including researchers from three different organizations. The roadmap is developed in an iterative process and is based on (1) systematic literature reviews of the area; (2) investigations of the state of practice, including a case survey and a survey; and (3) development and evaluation of solutions for asset identification and selection. The research activities resulted in identifying 11 areas in need of research. The areas are grouped into two categories: areas enabling evidence‐based decision‐making and those related to supporting the decision‐making. The roadmap outlines research needs in these 11 areas. The research challenges and research directions presented in this roadmap are key areas for further research to support evidence‐based decision‐making for composite software.

Systematic Literature Reviews (SLRs) have been adopted within Software Engineering (SE) for more than a decade to provide meaningful summaries of evidence on several topics. Many of these SLRs are now potentially not fully up-to-date, and there are no standard proposals on how to update SLRs in SE. The objective of this paper is to propose guidelines on how to best search for evidence when updating SLRs in SE, and to evaluate these guidelines using an SLR that was not employed during the formulation of the guidelines. To propose our guidelines, we compare and discuss outcomes from applying different search strategies to identify primary studies in a published SLR, an SLR update, and two replications in the area of effort estimation. These guidelines are then evaluated using an SLR in the area of software ecosystems, its update and a replication. The use of a single iteration forward snowballing with Google Scholar, and employing as a seed set the original SLR and its primary studies is the most cost-effective way to search for new evidence when updating SLRs. Furthermore, the importance of having more than one researcher involved in the selection of papers when applying the inclusion and exclusion criteria is highlighted through the results. Our proposed guidelines formulated based upon an effort estimation SLR, its update and two replications, were supported when using an SLR in the area of software ecosystems, its update and a replication. Therefore, we put forward that our guidelines ought to be adopted for updating SLRs in SE.

Usage of software engineering research in industrial practice is a well-known challenge. Synthesis of knowledge from multiple research studies is needed to provide evidence-based decision-support for industry. The objective of this paper is to present a vision of how a knowledge translation framework may look like in software engineering research, in particular how to translate research evidence into practice by combining contextualized expert opinions with research evidence. We adopted the framework of knowledge translation from health care research, adapted and combined it with a Bayesian synthesis method. The framework provided in this paper includes a description of each step of knowledge translation in software engineering. Knowledge translation using Bayesian synthesis intends to provide a systematic approach towards contextualized, collaborative and consensus-driven application of research results. In conclusion, this paper contributes towards the application of knowledge translation in software engineering through the presented framework.

Knowledge translation is defined, in health sciences, as “the exchange, synthesis and ethically sound application of research results in practice”. The objective of this paper is to implement and conduct a feasibility evaluation of a knowledge translation framework in software engineering. We evaluated the outcome of the knowledge translation framework in an industrial setting, along with the effectiveness of the interventions undertaken as part of knowledge translation in a multi-case study. The results of the evaluation suggest that the practitioners perceive the knowledge translation framework to be valuable and useful. In conclusion, this paper contributes towards the reporting of a systematic implementation of knowledge translation and evaluating its use in software engineering.

[Context] Systematic Literature Reviews (SLRs) have been adopted within the Software Engineering (SE) domain for more than a decade to provide meaningful summaries of evidence on several topics. Many of these SLRs are now outdated, and there are no standard proposals on how to update SLRs in SE. [Objective] The goal of this paper is to provide recommendations on how to best to search for evidence when updating SLRs in SE. [Method] To achieve our goal, we compare and discuss outcomes from applying different search strategies to identifying primary studies in a previously published SLR update on effort estimation. [Results] The use of a single iteration forward snowballing with Google Scholar, and employing the original SLR and its primary studies as a seed set seems to be the most cost-effective way to search for new evidence when updating SLRs. [Conclusions] The recommendations can be used to support decisions on how to update SLRs in SE.

This paper introduces a decision canvas for capturing architecture decisions in software and systems engineering. The canvas leverages a dedicated taxonomy, denoted GRADE, meant for establishing the basics of the vocabulary for assessing and choosing architectural assets in the development of software-intensive systems. The canvas serves as a template for practitioners to discuss and document architecture decisions, i.e., capture, understand and communicate decisions among decision-makers and to others. It also serves as a way to reflect on past decision-making activities devoted to both tentative and concluding decisions in the development of software-intensive systems. The canvas has been assessed by means of preliminary internal and external evaluations with four scenarios. The results are promising as the canvas fulfills its intended objectives while satisfying most of the needs of the subjects participating in the evaluation.

[Background] Systematic Literature Reviews (SLRs) are one of the important pillars when employing an evidence-based paradigm in Software Engineering. To date most SLRs have been conducted using a search strategy involving several digital libraries. However, significant issues have been reported for digital libraries and applying such search strategy requires substantial effort. On the other hand, snowballing has recently arisen as a potentially more efficient alternative or complementary solution. Nevertheless, it requires a relevant seed set of papers. [Aims] This paper proposes and evaluates a hybrid search strategy combining searching in a specific digital library (Scopus) with backward and forward snowballing. [Method] The proposed hybrid strategy was applied to two previously published SLRs that adopted database searches. We investigate whether it is able to retrieve the same included papers with lower effort in terms of the number of analysed papers. The two selected SLRs relate respectively to elicitation techniques (not confined to Software Engineering (SE)) and to a specific SE topic on cost estimation. [Results] Our results provide preliminary support for the proposed hybrid search strategy as being suitable for SLRs investigating a specific research topic within the SE domain. Furthermore, it helps overcoming existing issues with using digital libraries in SE. [Conclusions] The hybrid search strategy provides competitive results, similar to using several digital libraries. However, further investigation is needed to evaluate the hybrid search strategy.

Context Large software development projects involve multiple interconnected teams, often spread around the world, developing complex products for a growing number of customers and users. Succeeding with large-scale software development requires access to an enormous amount of knowledge and skills. Since neither individuals nor teams can possibly possess all the needed expertise, the resource availability in a team's knowledge network, also known as social capital, and effective knowledge coordination become paramount. Objective In this paper, we explore the role of social capital in terms of knowledge networks and networking behavior in large-scale software development projects. Method We conducted a multi-case study in two organizations, Ericsson and ABB, with software development teams as embedded units of analysis. We organized focus groups with ten software teams and surveyed 61 members from these teams to characterize and visualize the teams’ knowledge networks. To complement the team perspective, we conducted individual interviews with representatives of supporting and coordination roles. Based on survey data, data obtained from focus groups, and individual interviews, we compared the different network characteristics and mechanisms that support knowledge networks. We used social network analysis to construct the team networks, thematic coding to identify network characteristics and context factors, and tabular summaries to identify the trends. Results Our findings indicate that social capital and networking are essential for both novice and mature teams when solving complex, unfamiliar, or interdependent tasks. Network size and networking behavior depend on company experience, employee turnover, team culture, need for networking, and organizational support. A number of mechanisms can support the development of knowledge networks and social capital, for example, introduction of formal technical experts, facilitation of communities of practice and adequate communication infrastructure. Conclusions Our study emphasizes the importance of social capital and knowledge networks. Therefore, we suggest that, along with investments into training programs, software companies should also cultivate a networking culture to strengthen their social capital, a known driver of better performance.

Systematic literature studies have become standard practice in software engineering to synthesize evidence in different areas of the discipline. As more such studies are published, there is also a need to extend previously published systematic literature studies to cover new research papers. These first extensions become second-generation systematic literature studies. It has been asserted that snowballing would be a suitable search strategy for these types of second-generation studies, since newer studies ought to refer to previous research on a topic, and in particular to systematic literature studies published in an area. This paper compares using a snowballing search strategy with a published second-generation study using a database search strategy in the area of cross-company vs. within-company effort estimation. It is concluded that the approaches are comparable when it comes to which papers they find, although the snowballing approach is more efficient in this particular case.

Empirical studies of different kinds are nowadays regularly published in software engineering journals and conferences. Many empirical studies have been published, but are this sufficient? Individual studies are important, but the actual potential in relation to evidence-based software engineering [1] is not fully exploited. As a discipline we have to be able to go further to make our individual studies more useful. Other research should be able to leverage on the studies and industry should be able to make informed decisions based on the empirical research.

Systematic literature reviews in software engineering are necessary to synthesize evidence from multiple studies to provide knowledge and decision support. However, synthesis methods are underutilized in software engineering research. Moreover, translation of synthesized data (outcomes of a systematic review) to provide recommendations for practitioners is seldom practiced. The objective of this paper is to introduce the use of Bayesian synthesis in software engineering research, in particular to translate research evidence into practice by providing the possibility to combine contextualized expert opinions with research evidence. We adopted the Bayesian synthesis method from health research and customized it to be used in software engineering research. The proposed method is described and illustrated using an example from the literature. Bayesian synthesis provides a systematic approach to incorporate subjective opinions in the synthesis process thereby making the synthesis results more suitable to the context in which they will be applied. Thereby, facilitating the interpretation and translation of knowledge to action/application. None of the synthesis methods used in software engineering allows for the integration of subjective opinions, hence using Bayesian synthesis can add a new dimension to the synthesis process in software engineering research.

To facilitate the sharing and combination of knowledge by Global Software Engineering (GSE) researchers and practitioners, the need for a common terminology and knowledge classification scheme has been identified, and as a consequence, a taxonomy and an extension were proposed. In addition, one systematic literature review and a survey on respectively the state of the art and practice of effort estimation in GSE were conducted, showing that despite its importance in practice, the GSE effort estimation literature is rare and reported in an ad-hoc way. Therefore, this paper proposes a specialized GSE taxonomy for effort estimation, which was built on the recently proposed general GSE taxonomy (including the extension) and was also based on the findings from two empirical studies and expert knowledge. The specialized taxonomy was validated using data from eight finished GSE projects. Our effort estimation taxonomy for GSE can help both researchers and practitioners by supporting the reporting of new GSE effort estimation studies, i.e. new studies are to be easier to identify, compare, aggregate and synthesize. Further, it can also help practitioners by providing them with an initial set of factors that can be considered when estimating effort for GSE projects.

Abstract Context: Global software development (GSD) projects can have distributed teams that work independently in different locations or team members that are dispersed. The various development settings in GSD can influence quality during product evolution. When evaluating quality using defects as a proxy, the development settings have to be taken into consideration. Objective: The aim is to provide a systematic method for supporting investigations of the implication of GSD contexts on defect data as a proxy for quality. Method: A method engineering approach was used to incrementally develop the proposed method. This was done through applying the method in multiple industrial contexts and then using lessons learned to refine and improve the method after application. Results: A measurement instrument and visualization was proposed incorporating an understanding of the release history and understanding of GSD contexts. Conclusion: The method can help with making accurate inferences about development settings because it includes details on collecting and aggregating data at a level that matches the development setting in a GSD context and involves practitioners at various phases of the investigation. Finally, the information that is produced from following the method can help practitioners make informed decisions when planning to develop software in comparable circumstances. Copyright © 2016 John Wiley & Sons, Ltd.

Background: Systematic literature studies are commonly used in software engineering. There are two main ways of conducting the searches for these type of studies; they are snowballing and database searches. In snowballing, the reference list (backward snowballing - BSB) and citations (forward snowballing - FSB) of relevant papers are reviewed to identify new papers whereas in a database search, different databases are searched using predefined search strings to identify new papers. Objective: Snowballing has not been in use as extensively as database search. Hence it is important to evaluate its efficiency and reliability when being used as a search strategy in literature studies. Moreover, it is important to compare it to database searches. Method: In this paper, we applied snowballing in a literature study, and reflected on the outcome. We also compared database search with backward and forward snowballing. Database search and snowballing were conducted independently by different researchers. The searches of our literature study were compared with respect to the efficiency and reliability of the findings. Results: Out of the total number of papers found, snowballing identified 83% of the papers in comparison to 46% of the papers for the database search. Snowballing failed to identify a few relevant papers, which potentially could have been addressed by identifying a more comprehensive start set. Conclusion: The efficiency of snowballing is comparable to database search. It can potentially be more reliable than a database search however, the reliability is highly dependent on the creation of a suitable start set.

There exists no generally accepted theory in software engineering, and at the same time a scientific discipline needs theories. Some laws, hypotheses and conjectures exist, but yet no generally accepted theory. Several researchers and initiatives emphasize the need for theory in the discipline. The objective of this paper is to formulate a theory of software engineering. The theory is generated from empirical observations of industry practice, including several case studies and many years of experience in working closely between academia and industry. The theory captures the balancing of three different intellectual capitals: human, social and organizational capitals, respectively. The theory is formulated using a method for building theories in software engineering. It results in a theory where the relationships between the three different intellectual capitals are explored and explained. The theory is illustrated based on an industrial case study, where it is shown how decisions made in industry practice are explainable with the formulated theory, and the consequences of the decisions are made explicit. Based on the positive results, it is concluded that the theory may have a good explanatory power, although more evaluations are needed.

Background: Systematic literature studies have become common in software engineering, and hence it is important to understand how to conduct them efficiently and reliably.

Context: Systematic literature reviews have become common in software engineering in the last decade, but challenges remain.

Many companies have development sites around the globe. This inevitably means that development work may be transferred between the sites. This paper defines a classification of software transfer types, it divides transfers into three main types: full, partial and gradual transfers to describe the context of a transfer. The differences between transfer types, and hence the need for a classification, are illustrated with staffing curves for two different transfer types. The staffing curves are obtained through a combination of interviews with both high-level management and a group of experts, and an industrial case study. From the empirical work, it is concluded that the distribution of personnel differs for different types of transfer, which means that it is crucial to be clear about different classes of software transfers. If not, it is easy to underestimate the effort needed to transfer software work as well as additional costs related to the transfer as such.

In this paper we investigate the current state of work regarding alignment of Business, Architecture, Process, and Organisation (BAPO) perspectives in a software product development context. We planned to do that by conducting a systematic literature study to capture the state of the art in alignment of BAPO in software development. But, as it turned out we found that almost no substantial information is available about the alignment of BAPO in software development. Thus, based on the available literature and a small qualitative study, we defined a conceptual model of the alignment of BAPO including five levels of alignment that can be used as a basis for future empirical studies.

Systematic studies of the literature can be done in different ways. In particular, different guidelines propose different first steps in their recommendations, e.g. start with search strings in different databases or start with the reference lists of a starting set of papers.

SUMMARY Agile practices have received attention from industry as an alternative to plan‐driven software development approaches. Agile encourages, for example, small self‐organized collocated teams, whereas global software engineering (GSE) implies distribution across cultural, temporal, and geographical boundaries. Hence, combining them is a challenge. A systematic review was conducted to capture the status of combining agility with GSE. The results were limited to peer‐reviewed conference papers or journal articles, published between 1999 and 2009. The synthesis was made through classifying the papers into different categories (e.g. publication year, contribution type, research method). At the end, 81 papers were judged as primary for further analysis. The distribution of papers over the years indicated that GSE and Agile in combination has received more attention in the last 5 years. However, the majority of the existing research is industrial experience reports in which Agile practices were modified with respect to the context and situational requirements. The emergent need in this research area is suggested to be developing a framework that considers various factors from different perspectives when incorporating Agile in GSE. Practitioners may use it as a decision‐making basis in early phases of software development. Copyright © 2011 John Wiley & Sons, Ltd.

SUMMARY Globalization has influenced the way software is developed today, and many software organizations have started to actively utilize resources from around the world. However, these endeavors are recognized as very challenging, and they have attracted a lot of attention in software research in the past decade. Unlike many other research initiatives, which explore the complexities of distributed software development activities, the focus of this paper is on software transfers. Software transfers refer to activities that are moved from one location to another. The authors draw attention to the lessons learned from an empirical investigation of two transfer projects conducted at Ericsson. Both transfers were performed between a site in Sweden and a site in India. The observations outline a set of generic practices that have been found useful for transferring software development within a company. It also highlights a number of challenges to be addressed and a set of corresponding recommendations. Finally, the paper emphasizes the need to identify software products that are suitable for transfers and the need to monitor the long‐term effects of transfer. Copyright © 2011 John Wiley & Sons, Ltd.

Background: Software quality issues are commonly reported when off shoring software development. Value-based software engineering addresses this by ensuring key stakeholders have a common understanding of quality. Aim: This work seeks to understand the levels of alignment between key stakeholders on aspects of software quality for two products developed as part of an offshore in sourcing arrangement. The study further aims to explain the levels of alignment identified. Method: Representatives of key stakeholder groups for both products ranked aspects of software quality. The results were discussed with the groups to gain a deeper understanding. Results: Low levels of alignment were found between the groups studied. This is associated with insufficiently defined quality requirements, a culture that does not question management and conflicting temporal reflections on the product's quality. Conclusion: The work emphasizes the need for greater support to align success-critical stakeholder groups in their understanding of quality when off shoring software development.

Abstract Responsiveness to customer needs is an important goal in agile and lean software development. One major aspect is to have a continuous and smooth flow that quickly delivers value to the customer. In this paper we apply cumulative flow diagrams to visualize the flow of lean software development. The main contribution is the definition of novel measures connected to the diagrams to achieve the following goals: (1) increase throughput and reduce lead‐time to achieve high responsiveness to customers' needs and (2) to provide a tracking system that shows the progress/status of software product development. An evaluation of the measures in an industrial case study showed that practitioners found them useful and identify improvements based on the measurements, which were in line with lean and agile principles. Furthermore, the practitioners found the measures useful in seeing the progress of development for complex products where many tasks are executed in parallel. The measures are now an integral part of the improvement work at the studied company. Copyright © 2010 John Wiley & Sons, Ltd.

This paper presents the results of systematically reviewing the current research literature on the use of agile practices and lean software development in global software engineering (GSE). The primary purpose is to highlight under which circumstances they have been applied efficiently. Some common terms related to agile practices (e.g. scrum, extreme programming) were considered in formulating the search strings, along with a number of alternatives for GSE such as offshoring, outsourcing, and virtual teams. The results were limited to peer-reviewed conference papers/journal articles, published between 1999 and 2009. The synthesis was made through classifying the papers into different categories (e.g. research type, distribution). The analysis revealed that in most cases agile practices were modified with respect to the context and situational requirements. This indicates the need for future research on how to integrate all experiences and practices in a way to assist practitioners when setting up non-collocated agile projects.

Although global software work nowadays is not a phenomenon, research and practice is still addressing the complexities associated with the new forms of work enabled through globalization. `Go global' strategies usually prescribe two alternative approaches: distribution of software development activities across several locations, or re-location of work to another site. This research paper focuses on the latter approach that is called software product transfers and discusses findings from an empirical case study conducted in Ericsson. The observations indicate that transferring software work from one site to another site of the same company is a challenging endeavor. Furthermore, practices used in co-located projects do not necessarily lead to a desired outcome. The paper provides an overview of the challenges and highlights practical advice for handling software product transfers.

The long-term sustainability of a software product depends on more than developing features. Priorities are placed on aspects that support the development of software, like software product quality (eg. ISO 9126), project constraints - time and cost, and even the development of intellectual capital (IC). A greater focus on any one aspect takes priority from another, but as each aspects delivers a different type of value managers have trouble comparing and balancing these aspects. This paper presents a method to help determine the balance between key priorities in the software development process. The method is applied to a new case study, that also combines with results from previous studies. The results show it is possible to compare features, quality, time, cost and IC in a comprehensive way, with the case study showing that participants perceive a change from a shorter-term product perspective to a longer-term organisation beneficial to the business.

In order to draw valid conclusions when aggregating evidence it is important to describe the context in which industrial studies were conducted. This paper structures the context for empirical industrial studies and provides a checklist. The aim is to aid researchers in making informed decisions concerning which parts of the context to include in the descriptions. Furthermore, descriptions of industrial studies were surveyed.

Reading techniques help to guide reviewers during individual software inspections. In this experiment, we completely transfer the principle of statistical usage testing to inspection reading techniques for the first time. Statistical usage testing relies on a usage profile to determine how intensively certain parts of the system shall be tested from the users' perspective. Usage-based reading applies statistical usage testing principles by utilizing prioritized use cases as a driver for inspecting software artifacts (e.g., design). In order to reflect how intensively certain use cases should be inspected, time budgets are introduced to usage-based reading where a maximum inspection time is assigned to each use case. High priority use cases receive more time than low priority use cases. A controlled experiment is conducted with 23 Software Engineering M.Sc. students inspecting a design document. In this experiment, usage-based reading without time budgets is compared with time controlled usage-based reading. The result of the experiment is that time budgets do not significantly improve inspection performance. In conclusion, it is sufficient to only use prioritized use cases to successfully transfer statistical usage testing to inspections.

Quantitative failure estimates for software systems are traditionally made at end of testing using software reliability growth modeling. A persistent problem with most kinds of failure estimation methods and models is the dependency on historical data. This paper presents a method for estimating the total amount of failures possible to provoke from a unit, without historical data dependency. The method combines the results from having several developers testing the same unit with capture-recapture models to create an estimate of 'remaining' number of failures. The evaluation of the approach consists of two steps: first a pre-study where the tools and methods are tested in a large open source project, followed by an add-on to a project at a medium sized software company. The evaluation was a success. An estimate was created, and it can be used both as a quality gatekeeper for units and input to functional and system testing.

An important task in Requirements Engineering is to select which requirements that should go into a specific release of a system. This is a complex decision that requires balancing multiple perspectives against each other. In this article we investigate what students imagine is important to professionals in requirements selection. The reason for this is to understand whether the students are able to picture what industry professionals value, and whether the courses allow them to picture the state of industry practice. The results indicate that students have a good understanding of the way industry acts in the context of requirements selection, and students may work well as subjects in empirical studies in this area.

Increased popularity of global software engineering (GSE) has resulted in quite a number of research and industrial studies. As the area matures, an increased focus on empirically supported results leads to a greater potential impact on future research and industrial practice. However, since GSE scenarios are diverse, what works in one context might not directly apply in another. Thus it is necessary to understand, how GSE-related empirical findings should be reported to be useful for practitioners and researchers. Furthermore, itdasias important to summarize progress and get the big picture of published research to identify gaps and commonalities. In this paper we analyze differentiating factors of GSE scenarios and offer a classification scheme for describing the context of a GSE study. In addition, we report initial results of a systematic review on GSE-related empirical literature using papers from ICGSE 2006 and 2007, at the same time illustrating and evaluating the proposed scheme.

Change impact analysis is a change management activity that previously has been studied much from a technical perspective. For example, much work focuses on methods for determining the impact of a change. In this paper, we present results from a study on the role of impact analysis in the change management process. In the study, impact analysis issues were prioritised with respect to criticality by software professionals from an organisational perspective and a self-perspective. The software professionals belonged to three organisational levels: operative, tactical and strategic. Qualitative and statistical analyses with respect to differences between perspectives as well as levels are presented. The results show that important issues for a particular level are tightly related to how the level is defined. Similarly, issues important from an organisational perspective are more holistic than those important from a self-perspective. However, our data indicate that the self-perspective colours the organisational perspective, meaning that personal opinions and attitudes cannot easily be disregarded. In comparing the perspectives and the levels, we visualise the differences in a way that allow us to discuss two classes of issues: high-priority and medium-priority. The most important issues from this point of view concern fundamental aspects of impact analysis and its execution.

Recent research in software engineering has highlighted the need to ensure alignment between business objectives, customer requirements and product development. If the business is to meet its strategic objectives, the Requirements Engineering (RE) activities must be executed in a manner such that they support these higher level objectives. A clear alignment between RE activities and the strategic objectives of the organization should underscore the merit of IT investment and the opportunities for competitive advantage that can be pursued as a result by the organization. This research begins with a detailed investigation of the complexity of decision-making during RE activities on business, product and project levels. Secondly, it investigates "in-project" level RE decisions and provides empirical findings from an industrial case study. The findings show that RE project related decisions are influenced by business and product requirements. Although the software developers in this study recognized the importance of developing software products that meet customer requirements and that are aligned with business objectives, they found it difficult to achieve this when the customer was external and the communication was a major problem. A lack of understanding the business strategy and a lack of communication between business, product and project level stakeholders were among the problems that are highlighted by this study.

There is a need to identify factors that affect the result of empirical studies in software engineering research. It is still the case that seemingly identical replications of controlled experiments result in different conclusions due to the fact that all factors describing the experiment context are not clearly defined and hence controlled. In this article, a scheme for describing the participants of controlled experiments is proposed and evaluated. It consists of two main factors, the incentives for participants in the experiment and the experience of the participants. The scheme has been evaluated by classifying a set of previously conducted experiments from literature. It can be concluded that the scheme was easy to use and understand. It is also found that experiments that are classified in the same way to a large extent point at the same results, which indicates that the scheme addresses relevant factors.

Abstract Software process improvement is a challenge in general and in particular for small‐ and medium‐sized companies. Assessment is one important step in improvement. However, given that a list of improvement issues has been derived, it is often very important to be able to prioritize the improvement proposals and also look at the potential dependencies between them. This paper comes from an industrial need to enable prioritization of improvement proposals and to identify their dependencies. The need was identified in a small‐ and medium‐sized software development company. Based on the need, a method for prioritization and identification of dependencies of improvement proposals was developed. The prioritization part of the method is based on a multi‐decision criteria method and the dependencies are identified using a dependency graph. The developed method has been successfully applied in the company, where people with different roles applied the method. The paper presents both the method as such and the successful application of it. It is concluded that the method worked as a means for prioritization and identification of dependencies. Moreover, the method also allowed the employees to discuss and reason about the improvement actions to be taken in a structured and systematic way. Copyright © 2004 John Wiley & Sons, Ltd.

To sustain the qualities of a software system during evolution, and to adapt the quality attributes as the requirements evolve, it is necessary to have a clear software architecture that is understood by all developers and to which all changes to the system adheres. This software architecture can be created beforehand, but must also be updated as the domain of the software, and hence the requirements on the software system evolve. Creating a software architecture for a system or part of a system so that the architecture fulfils the desired quality requirements is often hard. In this paper we propose a decision-support method to aid in the understanding of different architecture candidates for a software system. We propose a method that is adaptable with respect to both the set of potential architecture candidates and quality attributes relevant for the system's domain to help in this task. The method creates a support framework, using a multi-criteria decision method, supporting comparison of different software architecture candidates for a specific software quality attribute and vice versa, and then uses this support framework to reach a consensus on the benefits and liabilities of the different software architecture candidates and to increase the confidence in the resulting architecture decision.

Software quality can be defined as the customers' perception of how a system works. Inspection is a method to monitor and control the quality throughout the development cycle. Reading techniques applied to inspections help reviewers to stay focused on the important parts of an artifact when inspecting. However, many reading techniques focus on finding as many faults as possible, regardless of their importance. Usage-based reading helps reviewers to focus on the most important parts of a software artifact from a user's point of view. We present an experiment, which compares usage-based and checklist-based reading. The results show that reviewers applying usage-based reading are more efficient and effective in detecting the most critical faults from a user's point of view than reviewers using checklist-based reading. Usage-based reading may be preferable for software organizations that utilize or start utilizing use cases in their software development.

To sustain the qualities of a software system during evolution, and to adapt the quality attributes as the requirements evolve, it is necessary to have a clear software architecture that is understood by all developers and to which all changes to the system adheres. This software architecture can be created beforehand, but must also be updated as the domain of the software, and hence the requirements on the software system evolves. Creating an architectural structure for a system or part of a system so that the architecture fulfils the desired quality requirements is often hard. In this paper we propose a decision support method to aid in the understanding of different architecture structure candidates for a software system. We propose a method that is adaptable with respect to both the set of potential architecture structures, and quality attributes relevant for the system's domain to help in this task. The method creates a support framework, using a multi-criteria decision method, supporting comparison of different software architecture structures for a specific software quality attribute and vice versa. Moreover, given a prioritization of quality attributes for the software system, or part thereof, the most suitable software architecture structure can be indicated using the created framework.

Project success is influenced by many factors. Some are primary drivers of project success, others secondary. However, these secondary factors may be no less important to understand and learn from than the primary factors. In addition, project success drivers are often qualitative and subjective, eluding analysis through traditional statistical methods. This paper presents a method that analyses the influence and nature of primary and lower order project success drivers. A case study illustrates the usefulness of this analysis and the additional understanding gained from including lower order success drivers in the analysis. The method extends existing work that has been restricted to primary drivers.

Abstract Software inspections, which were originally developed by Michael Fagan in 1976, are an important means to verify and achieve sufficient quality in many software projects today. Since Fagan's initial work, the importance of software inspections has been long recognized by software developers and many organizations. Various proposals have been made by researchers in the hope of improving Fagan's inspection method. The proposals include structural changes to the process and several types of support for the inspection process. Most of the proposals have been empirically investigated in different studies. This is a review paper focusing on the software inspection process in the light of Fagan's inspection method and it summarizes and reviews other types of software inspection processes that have emerged in the last 25 years. This paper also addresses important issues related to the inspection process and examines experimental studies and their findings that are of interest with the purpose of identifying future avenues of research in software inspection. Copyright © 2002 John Wiley & Sons, Ltd.

Abstract Components that have defects after release, but not during testing, are very undesirable as they point to ‘holes’ in the testing process. Either new components were not tested enough, or old ones were broken during enhancements and defects slipped through testing undetected. The latter is particularly pernicious, since customers are less forgiving when existing functionality is no longer working than when a new feature is not working quite properly. Rather than using capture–recapture models and curve‐fitting methods to estimate the number of remaining defects after inspection, these methods are adapted to estimate the number of components with post‐release defects that have no defects in testing. A simple experience‐based method is used as a basis for comparison. The estimates can then be used to make decisions on whether or not to stop testing and release software. While most investigations so far have been experimental or have used virtual inspections to do a statistical validation, the investigation presented in this paper is a case study. This case study evaluates how well the capture–recapture, curve‐fitting and experience‐based methods work in practice. The results show that the methods work quite well. A further benefit of these techniques is that they can be applied to new systems for which no historical data are available and to releases that are very different from each other. Copyright © 2002 John Wiley & Sons, Ltd.

Software systems evolve over time and it is often difficult to maintain them. One reason for this is that often it is hard to understand the previous release. Further, even if architecture and design models are available and up to date, they primarily represent the functional behavior of the system. To evaluate whether it is possible to also represent some nonfunctional aspects, an experiment has been conducted. The objective of the experiment is to evaluate the cognitive suitability of some visual representations that can be used to represent a control relation, software component size and component external and internal complexity. Ten different representations are evaluated in a controlled environment using 35 subjects. The results from the experiment show that representations with low cognitive accessibility weight can be found. In an example, these representations are used to illustrate some qualities in an SDL block diagram. It is concluded that the incorporation of these representations in architecture and design descriptions is both easy and probably worthwhile. The incorporation of the representations should enhance the understanding of previous releases and, hence, help software developers in evolving and maintaining complex software systems.

Estimation of the defect content is important to enable quality control throughout the software development process. Capture-recapture methods and curve fitting methods have been suggested as tools to estimate the defect content after a review. The methods are highly reliant on the quality of the data. If the number of reviewers is fairly small, it becomes difficult or even impossible to get reliable estimates. This paper presents a comprehensive study of estimates based on two reviewers, using real data from reviews. Three experience-based defect content estimation methods are evaluated vs. methods that use data only from the current review. Some models are possible to distinguish from each other in terms of statistical significance. In order to gain an even better understanding, the best models are compared subjectively. It is concluded that the experience-based methods provide some good opportunities to estimate the defect content after a review.

Abstract Many of today's software systems evolve through a series of releases that add new functionality and features, in addition to the results of corrective maintenance. As the systems evolve over time it is necessary to keep track of and manage their problematic components. Our focus is to track system evolution and to react before the systems become difficult to maintain. To do the tracking, we use a method based on a selection of statistical techniques. In the case study we report here that had historical data available primarily on corrective maintenance, we apply the method to four releases of a system consisting of 130 components. In each release, components are classified as fault‐prone if the number of defect reports written against them are above a certain threshold. The outcome from the case study shows stabilizing principal components over the releases, and classification trees with lower thresholds in their decision nodes. Also, the variables used in the classification trees' decision nodes are related to changes in the same files. The discriminant functions use more variables than the classification trees and are more difficult to interpret. Box plots highlight the findings from the other analyses. The results show that for a context of corrective maintenance, principal components analysis together with classification trees are good descriptors for tracking software evolution. Copyright © 2001 John Wiley & Sons, Ltd.

As software systems evolve over a series of releases, it becomes important to know which components are stable compared to components that show repeated need for corrective maintenance. To track these across multiple releases, we adapt a reverse architecting technique to defect reports of a series of releases. Fault relationships among system components are identified based on whether they are involved in the same defect report, and for how many defect reports this occurs. There are degrees of fault-coupling between components depending on how often these components are involved in a defect fix. After these fault-coupling relationships between components are extracted, they are abstracted to the subsystem level. We also identify a measure for fault cohesion (i.e. fault-proneness of components locally.) The resulting fault architecture figures show for each release what its most fault-prone relationships are. Comparing across releases makes it possible to see whether some relationships between components are repeatedly fault-prone, indicating an underlying systemic architecture problem. We illustrate our technique on a large commercial system consisting of over 800 KLOC of C, C++, and microcode. Copyright © 2000 John Wiley & Sons, Ltd.

This paper presents a collection of experience related to success factors in graduate and postgraduate education. The experiences are mainly concerned with how to make the education relevant from an industrial viewpoint. This is emphasized as a key issue in software engineering education and research, since the main objective is to give the students a good basis for large-scale software development in an industrial environment. The presentation is divided into experiences at the graduate and postgraduate levels respectively. For each level a number of strategies to achieve industrial relevance are presented. These strategies have been successful, but it is concluded that more can be done regarding industrial collaboration in the planning and conduction of experiments and case studies. Another interesting strategy for the future is a special postgraduate programme for people employed in industry.

As software systems mature, there is the danger that not only code decays, but software architectures as well. We adapt a reverse architecting technique to defect reports of a series of releases. Relationships among system components are identified based on whether they are involved in the same defect report, and for how many defect reports this occurs. There are degrees of fault-coupling between components depending on how often two components are involved in a defect fix. After these fault-coupling relationships between components are extracted, they are abstracted to the subsystem level. The resulting fault architecture figures show for each release what its most fault-prone relationships are. Comparing across releases shows whether some relationships between components are repeatedly fault prone, indicating an underlying systemic architecture problem. We illustrate our technique on a large commercial system consisting of over 800 KLOC of C, C++, and microcode.

Capture-recapture models and curve-fitting models have been proposed to estimate the remaining number of defects after a review. This estimation gives valuable information to monitor and control software reliability. However, the different models provide different estimates making it difficult to know which estimate is the most accurate. One possible solution is to, as in this paper, focus on different opportunities to estimate intervals. The study is based on thirty capture-recapture data sets from software reviews. Twenty of the data sets are used to create different models to perform estimation. The models are then evaluated on the remaining ten data sets. The study shows that the use of historical data in model building is one way to overcome some of the problems experienced with both capture-recapture and curve-fitting models, to estimate the defect content after a review.

The required effort of a task can be estimated subjectively in interviews with experts in an organization in different ways. Interview techniques dealing with which type of questions to ask are evaluated and techniques for combining estimates from individuals into one estimate are compared in an experiment. The result shows that the interview technique is not as important as the combination technique. The estimate which is best with respect to mean value and standard deviation of the effort is based on an equal weighting of all individual estimates. The experiment is performed within the Personal Software Process (PSP).

Reviews are essential for defect detection and they provide an opportunity to control the software development process. This paper focuses upon methods for estimating the defect content after a review and hence to provide support for process control. Two new estimation methods are introduced as the assumptions of the existing statistical methods are not fulfilled. The new methods are compared with a maximum-likelihood approach. Data from several reviews are used to evaluate the different methods. It is concluded that the new estimation methods provide new opportunities to estimate the defect content.

Software systems are often getting older than expected, and it is a challenge to try to make sure that they grow old gracefully. This implies that methods are needed to ensure that system components are possible to maintain. In this paper, the need to investigate, classify and study software components is emphasized. A classification method is proposed. It is based on classifying the software components into green, yellow and red components. The classification scheme is complemented with a discussion of suitable models to identify problematic components. The scheme and the models are illustrated in a minor case study to highlight the opportunities. The long term objective of the work is to define methods, models and metrics which are suitable to use in order to identify software components which have to be taken care of through either tailored processes (e.g. additional focus on verification and validation) or reengineering. The case study indicates that the long term objective is realistic and worthwhile.

Reuse is pin-pointed as a key factor to improve productivity and reliability of software systems. Verification and validation of software components and the resulting system is important for reuse to be beneficial on a broad industrial basis. The paper suggests a modelling approach which is suitable for reliability certification of modular systems. It discusses a general reliability certification procedure and provides guidelines and opportunities for how to certify software components and also presents some alternatives for certification of modular systems. We conclude that to create reliable systems, we must start certifying the individual constituents of the systems.

There is a general need to understand software defects and the ability to detect defects during different activities. This is particularly important in relation to software process improvement, where one objective may be to decrease the number of defects. The Personal Software Process (PSP) has gained attention during the last couple of years as a way to individual improvement in software development. Thus, the PSP is an interesting starting point to understand software defects and in particular the detection process. The paper presents a study of software defect detection for 59 students taking a PSP course. In summary, the study provides valuable insight into software defect detection in the PSP. Some of the results are interesting not only for the PSP, but from a general perspective in the understanding of software defect detection. The understanding of software defects forms the basis for improving the reliability of software.

This paper describes a final-year Master's course in large-scale software development. A number of issues such as closeness to an application domain and executing the software on a real target system are stressed in order to imitate an industrial environment. The experience gained from the course is discussed. In particular, effort data (in terms of man-hours) from the course are presented, both from the plan devised by the students, and the actual outcome. Furthermore, it is discussed how the data can be used to create an experience base for the future. The objective for next year's course is to let the students plan their projects based on the experience base. The experience base will also form the basis to complement the course with a complete experience factory.

The traditional approach of reliability prediction using software reliability growth models requires a large number of failures which might not be available at the beginning of the testing. The commonly used maximum likelihood estimates may not even exist or converge to a reasonable value. In this paper, an approach of making use of information from similar projects in order to obtain an early estimation of one model parameter for a current project is studied. As most of the two-parameter reliability growth models contains one parameter related to the number of faults in the software and a reliability growth rate parameter related to the testing efficiency, information from a similar project can used to estimate the reliability growth rate parameter and the limited failure data from initial testing is used to estimate the other parameter. Our case study shows that this approach is very easy to use as the estimation does not require a numerical algorithm and it always exists. It is also very stable and when the maximum likelihood estimates exist and are reasonable, our approach gives values very close to that, and the approximate confidence interval is overlapping for most cases.

Software complexity measures are often proposed as suitable indicators of different software quality attributes. The paper presents a study of some complexity measures and their correlation with the number of failure reports, and a number of problems are identified. The measures are such poor predictors that one may as well use a very simple measure. The proposed measure is supposed to be ironic to stress the need for a more scientific approach to software measurement. The objective is primarily to encourage discussions concerning methods to estimate different quality attributes. It is concluded that either completely new methods are needed to predict software quality attributes or a new view on predictions from complexity measures is needed. This is particularly crucial if the software industry is to use software metrics successfully on a broad basis.

Most software reliability models are applicable to a single piece of software. For more complex systems, Markov models have been studied assuming that complete reliability information at the module level is available. We study an additive model which assumes that each subsystem or module undergoes independent testing and the reliability of the complete system has to be assessed. Subsystem reliabilities are estimated and the system reliability is assessed using the additive model. The approach is simple, but generic in a sense that existing models can be combined. An example is also presented based on the log-power-model, a model with simple graphical interpretation and it shows that the modular information should be used for system reliability assessment whenever such information is available.

Abstract The use of capture‐recapture to estimate the residual faults in a software artifact has evolved as a promising method. However, the assumptions needed to make the estimates are not completely fulfilled in software development, leading to an underestimation of the residual fault content. Therefore, a method employing a filtering technique with an experience factor to improve the estimate of the residual faults is proposed in this paper. An experimental study of the capture‐recapture method with this correction method has been conducted. It is concluded that the correction method improves the capture‐recapture estimate of the number of residual defects in the inspected document.

Reuse is becoming one of the key areas in dealing with the cost and quality of software systems. An important issue is the reliability of the components, hence making certification of software components a critical area. The objective of this article is to try to describe methods that can be used to certify and measure the ability of software components to fulfil the reliability requirements placed on them. A usage modelling technique is presented, which can be used to formulate usage models for components. This technique will make it possible not only to certify the components, but also to certify the system containing the components. The usage model describes the usage from a structural point of view, which is complemented with a profile describing the expected usage in figures. The failure statistics from the usage test form the input of a hypothesis certification model, which makes it possible to certify a specific reliability level with a given degree of confidence. The certification model is the basis for deciding whether the component can be accepted, either for storage as a reusable component or for reuse. It is concluded that the proposed method makes it possible to certify software components, both when developing for and with reuse.>

Software reliability engineering is not only the use of software reliability models and similar techniques, it is the use of sensible engineering principles with cost/benefit analysis throughout the software life cycle to obtain reliable software. The need to have a comprehensive view on software development to engineer reliable software is emphasized. Cleanroom Software Engineering is proposed as being the basis for developing reliable software. The paper in particular discusses some extensions to Cleanroom, both in terms of adaptations and additions. Particular emphasis is on high-level design techniques and methods for reliability certification of the software. The comprehensive view of software is supported by several success stores, both with references to results presented in literature as well as experiences from projects conducted by Q-Labs. The results obtained are encouraging. The methods proposed are shown to give a substantial gain in the development of reliable software.

Presents a method proposal for estimation of software reliability before the implementation phase. The method is based upon a formal description technique and that it is possible to develop a tool for performing dynamic analysis, i.e. locating semantic faults in the design. The analysis is performed by applying a usage profile as input as well as doing a full analysis, i.e. locating all faults that the tool can find. The tool must provide failure data in terms of time since the last failure was detected. The mapping of the dynamic failures to the failures encountered during statistical usage testing and operation is discussed. The method can be applied either on the software specification or as a step in the development process by applying it on the design descriptions. The proposed method allows for software reliability estimations that can be used both as a quality indicator, and also for planning and controlling resources, development times etc. at an early stage in the development of software systems.>