The increasing reliance on Open Source Software (OSS) in organizations’ software supply chains necessitates robust mechanisms in the intake process to ensure sourced components’ long-term viability and maintenance. Assessing OSS project health in the intake process is complex due to the wide range of socio-technical factors involved. This study aims to explore how the health of OSS projects may be assessed by practitioners from organizations’ intake perspective. We conducted a qualitative interview survey with 17 industry experts to identify aspects and related metrics of OSS health. These were mapped against literature and two existing industry frameworks. A subset was identified and applied through a case study at a large international automotive manufacturer. 21 health aspects with 72 connected metrics were identified covering community productivity and stability, orchestration, production processes, and outputs. Many metrics map against industry frameworks, while qualitative aspects are missing support. Special consideration is needed when assessing and comparing the health of OSS projects, including their life-cycle stage, complexity, governance concentration, and strategic importance for the focal organization. The case study shows that not all aspects and metrics may be leveraged due to resource constraints and complexity. Instead, subsets of metrics need to be prioritized, and applied in a structured approach using qualitative and quantitative means. The study provides a foundation and a starting point for developers in introducing health assessments of OSS components in their intake processes, while also pushing convergence towards a common corpus of health assessment in practice.

Abstract During software evolution, decisions related to components' origin or source significantly impact the quality properties of the product and development metrics such as cost, time to market, ease of maintenance, and further evolution. Thus, such decisions should ideally be supported by evidence, i.e., using previous experiences and information from different sources, even own previous experiences. A hindering factor to such reuse of previous experiences is that these decisions are highly context‐dependent and it is difficult to identify when previous experiences come from sufficiently similar contexts to be useful in a current setting. Conversely, when documenting a decision (as a decision experience), it is difficult to know which context factors will be most beneficial when reusing the experience in the future. An interview study is performed to identify a list of context factors that are perceived to be most important by practitioners when using experiences to support decision‐making for component sourcing, using a specific scenario with alternative sources of experiences. We observed that the further away (from a company or an interviewee) the experience evidence is, as is the case for online experiences, the more context factors are perceived as important by practitioners to make use of the experience. Furthermore, we discuss and identify further research to make this type of decision‐making more evidence‐based.

Motivation: Society’s dependence on Open Source Software (OSS) and the communities that maintain the OSS is ever-growing. So are the potential risks of, e.g., vulnerabilities being introduced in projects not actively maintained. By assessing an OSS project’s capability to stay viable and maintained over time without interruption or weakening, i.e., the OSS health, users can consider the risk implied by using the OSS as is, and if necessary, decide whether to help improve the health or choose another option. However, such assessment is complex as OSS health covers a wide range of sub-topics, and existing support is limited. Aim: We aim to create an overview of characteristics that affect the health of an OSS project and enable the assessment thereof. Method: We conduct a snowball literature review based on a start set of 9 papers, and identify 146 relevant papers over two iterations of forward and backward snowballing. Health characteristics are elicited and coded using structured and axial coding into a framework structure. Results: The final framework consists of 107 health characteristics divided among 15 themes. Characteristics address the socio-technical spectrum of the community of actors maintaining the OSS project, the software and other deliverables being maintained, and the orchestration facilitating the maintenance. Characteristics are further divided based on the level of abstraction they address, i.e., the OSS project-level specifically, or the project’s overarching ecosystem of related OSS projects. Conclusion: The framework provides an overview of the wide span of health characteristics that may need to be considered when evaluating OSS health and can serve as a foundation both for research and practice.

BACKGROUND: The COVID-19 pandemic triggered a natural experiment of an unprecedented scale as companies closed their offices and sent employees to work from home. Many managers were concerned that their engineers would not be able to work effectively from home, or lack the motivation to do so, and that they would lose control and not even notice when things go wrong. As many companies announced their post-COVID permanent remote-work or hybrid home/office policies, the question of what can be expected from software engineers who work from home becomes more and more relevant. AIMS: To understand the nature of home telework we analyze the evidence of perceived changes in productivity comparing office work before the pandemic with the work from home during the pandemic from thirteen empirical surveys of practitioners. METHOD: We analyzed data from six corporate surveys conducted in four Scandinavian companies combined with the results of seven published surveys studying the perceived changes in productivity in industrial settings. In addition, we sought explanations for the variation in perceived productivity among the engineers from the studied companies through the qualitative analysis of open-ended questions and interviews. RESULTS: Combined results of 7686 data points suggest that though on average perceived productivity has not changed significantly, there are developers who report being more productive, and developers being less productive when working from home. Positively affected individuals in some surveys form large groups of respondents (up to 50%) and mention benefiting from a better organization of work, increased flexibility and focus. Yet, there are equally large groups of negatively affected respondents (up to 51%) who complain about the challenges related to remote teamwork and collaboration, as well as emotional issues, distractions and poor home office environment and equipment. Finally, positive trends are found in longitudinal surveys, i.e., developers' productivity in the later months of the pandemic show better results than those in the earlier months. CONCLUSIONS: We conclude that behind the average "no change" lays a large variation of experiences, which means that the work from home might not be for everyone. Yet, a longitudinal analysis of the surveys is encouraging, as it shows that the more pessimistic results might be influenced by the initial experiences of an unprecedented crisis. At the end, we put forward the lessons learned during the pandemic that can inspire the new post-pandemic work policies.

Abstract Quality requirements deal with how well a product should perform the intended functionality, such as start-up time and learnability. Researchers argue they are important and at the same time studies indicate there are deficiencies in practice. Our goal is to review the state of evidence for quality requirements. We want to understand the empirical research on quality requirements topics as well as evaluations of quality requirements solutions. We used a hybrid method for our systematic literature review. We defined a start set based on two literature reviews combined with a keyword-based search from selected publication venues. We snowballed based on the start set. We screened 530 papers and included 84 papers in our review. Case study method is the most common (43), followed by surveys (15) and tests (13). We found no replication studies. The two most commonly studied themes are (1) differentiating characteristics of quality requirements compared to other types of requirements, (2) the importance and prevalence of quality requirements. Quality models, QUPER, and the NFR method are evaluated in several studies, with positive indications. Goal modeling is the only modeling approach evaluated. However, all studies are small scale and long-term costs and impact are not studied. We conclude that more research is needed as empirical research on quality requirements is not increasing at the same rate as software engineering research in general. We see a gap between research and practice. The solutions proposed are usually evaluated in an academic context and surveys on quality requirements in industry indicate unsystematic handling of quality requirements.

Abstract Software engineering is decision intensive. Evidence‐based software engineering is suggested for decision‐making concerning the use of methods and technologies when developing software. Software development often includes the reuse of software assets, for example, open‐source components. Which components to use have implications on the quality of the software (e.g., maintainability). Thus, research is needed to support decision‐making for composite software. This paper presents a roadmap for research required to support evidence‐based decision‐making for choosing and integrating assets in composite software systems. The roadmap is developed as an output from a 5‐year project in the area, including researchers from three different organizations. The roadmap is developed in an iterative process and is based on (1) systematic literature reviews of the area; (2) investigations of the state of practice, including a case survey and a survey; and (3) development and evaluation of solutions for asset identification and selection. The research activities resulted in identifying 11 areas in need of research. The areas are grouped into two categories: areas enabling evidence‐based decision‐making and those related to supporting the decision‐making. The roadmap outlines research needs in these 11 areas. The research challenges and research directions presented in this roadmap are key areas for further research to support evidence‐based decision‐making for composite software.

Component-based software engineering is a common approach in the development and evolution of contemporary software systems. Different component sourcing options are available, such as: (1) Software developed internally (in-house) , (2) Software developed outsourced , (3) Commercial off-the-shelf software , and (4) Open-Source Software . However, there is little available research on what attributes of a component are the most important ones when selecting new components. The objective of this study is to investigate what matters the most to industry practitioners when they decide to select a component. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using compositional data analysis. The results of this exploratory analysis showed that cost was clearly considered to be the most important attribute for component selection. Other important attributes for the practitioners were: support of the component , longevity prediction , and level of off-the-shelf fit to product . Moreover, several practitioners still consider in-house software development to be the sole option when adding or replacing a component. On the other hand, there is a trend to complement it with other component sourcing options and, apart from cost, different attributes factor into their decision. Furthermore, in our analysis, nonparametric tests and biplots were used to further investigate the practitioners’ inherent characteristics. It seems that smaller and larger organizations have different views on what attributes are the most important, and the most surprising finding is their contrasting views on the cost attribute: larger organizations with mature products are considerably more cost aware.

Component-based software engineering is a common approach to develop and evolve contemporary software systems where different component sourcing options are available: 1)Software developed internally (in-house), 2)Software developed outsourced, 3)Commercial of the shelf software, and 4) Open Source Software. However, there is little available research on what attributes of a component are the most important ones when selecting new components. The object of the present study is to investigate what matters the most to industry practitioners during component selection. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using Compositional Data Analysis. The descriptive results showed that Cost was clearly considered the most important attribute during the component selection. Other important attributes for the practitioners were: Support of the component, Longevity prediction, and Level of off-the-shelf fit to product. Next, an exploratory analysis was conducted based on the practitioners' inherent characteristics. Nonparametric tests and biplots were used. It seems that smaller organizations and more immature products focus on different attributes than bigger organizations and mature products which focus more on Cost.

The choice of which software component to use influences the success of a software system. Only a few empirical studies investigate how the choice of components is conducted in industrial practice. This is important to understand to tailor research solutions to the needs of the industry. Existing studies focus on the choice for off-the-shelf (OTS) components. It is, however, also important to understand the implications of the choice of alternative component sourcing options (CSOs), such as outsourcing versus the use of OTS. Previous research has shown that the choice has major implications on the development process as well as on the ability to evolve the system. The objective of this study is to explore how decision making took place in industry to choose among CSOs. Overall, 22 industrial cases have been studied through a case survey. The results show that the solutions specifically for CSO decisions are deterministic and based on optimization approaches. The non-deterministic solutions proposed for architectural group decision making appear to suit the CSO decision making in industry better. Interestingly, the final decision was perceived negatively in nine cases and positively in seven cases, while in the remaining cases it was perceived as neither positive nor negative.

This paper introduces a decision canvas for capturing architecture decisions in software and systems engineering. The canvas leverages a dedicated taxonomy, denoted GRADE, meant for establishing the basics of the vocabulary for assessing and choosing architectural assets in the development of software-intensive systems. The canvas serves as a template for practitioners to discuss and document architecture decisions, i.e., capture, understand and communicate decisions among decision-makers and to others. It also serves as a way to reflect on past decision-making activities devoted to both tentative and concluding decisions in the development of software-intensive systems. The canvas has been assessed by means of preliminary internal and external evaluations with four scenarios. The results are promising as the canvas fulfills its intended objectives while satisfying most of the needs of the subjects participating in the evaluation.

Abstract Software cost estimation is a key process in project management. Estimations in the initial project phases are made with a lot of uncertainty that influences estimation accuracy which typically increases as the project progresses in time. Project data collected during the various project phases can be used in a progressive time‐dependent fashion to train software cost estimation models. Our motivation is to reduce uncertainty and increase confidence based on the understanding of patterns of effort distributions in development phases of real‐world projects. In this work, we study effort distributions and suggest a four‐stage progressive software cost estimation model, adjusting the initial effort estimates during the development life‐cycle based on newly available data. Initial estimates are reviewed on the basis of the experience gained as development progresses and as new information becomes available. The proposed model provides an early, a post‐planning, a post‐specifications, and a post‐design estimate, while it uses industrial data from the ISBSG (R10) dataset. The results reveal emerging patterns of effort distributions and indicate that the model provides effective estimations and exhibits high explanatory value. Contributions in lessons learned and practical implications are also provided.

Efficient development of high quality software is tightly coupled to the ability of quickly taking complex decisions based on trustworthy facts. In component-based software engineering, the decisions related to selecting the most suitable component among functionally-equivalent ones are of paramount importance. Despite sharing the same functionality, components differ in terms of their extra-functional properties. Therefore, to make informed selections, it is crucial to evaluate extra-functional properties in a systematic way. To date, many properties and evaluation methods that are not necessarily compatible with each other exist. The property model ontology presented in this paper represents the first step towards providing a systematic way to describe extra-functional properties and their evaluation methods, and thus making them comparable. This is beneficial from two perspectives. First, it aids researchers in identifying comparable property models as a guide for empirical evaluations. Second, practitioners are supported in choosing among alternative evaluation methods for the properties of their interest. The use of the ontology is illustrated by instantiating a subset of property models relevant in the automotive domain.

This paper reports results from an independent observational study of an automotive software development research project. The study is carried out as a monitoring activity of the project, which is inexpensive but still representative of real automotive software development cases, thus providing the basis for more rigorous studies. The objective is to take initial steps to improve our understanding of architectural decision-making in the development of software in the automotive domain. The key findings summarize issues surfacing during the development process related to the problem articulation and formulation, the impact of participant experience, the definition of requirements, the decision process, and the effect of the decisions made on the system architecture evolution. The paper offers some insights that can be useful to gain understanding of how decisions are typically made in real settings, i.e., based on gut-feeling, which is important when designing decision support systems for architectural design decisions.

One of the most significant tasks of component-based software development is concerned with finding suitable components for integration. This paper introduces a novel development framework that promotes reusability and focuses on assessing the suitability level of candidate components. A specifications profile is first created using a semi-formal natural language that describes the desired functional and non-functional properties of the component(s) sought. A parser automatically recognizes parts of the profile and translates them into instance values of a dedicated CBSE ontology, the latter addressing issues of components' reusability. Available components on the market are also stored as instances of the CBSE ontology. Matching between required and offered component properties takes place automatically at the level of the ontology items and a suitability ratio is calculated that suggests which components to consider for integration.

An agile software development process is often claimed to increase productivity. However, productivity measurement in agile software development is little researched. Measures are not explicitly defined nor commonly agreed upon. In this paper, we highlight the agile productivity measures reported in literature by means of a research method called scoping study. We were able to identify 12 papers reporting the productivity measures in agile software development processes. We found that finding, understanding and putting into use agile productivity definitions is not an easy task. From the perspective of common roles in agile software development process and existing knowledge workers’ productivity dimensions, we also emphasize that none of the productivity measures satisfy these fully. We recommend that future effort should be focused on defining agile productivity in measurable, practicable and meaningful form.

Traditionally, Embedded Systems (ES) are tightly linked to physical products, and closed both for communication to the surrounding world and to additions or modifications by third parties. New technical solutions are however emerging that allow addition of plug-in software, as well as external communication for both software installation and data exchange. These mechanisms in combination will allow for the construction of Federated Embedded Systems (FES). Expected benefits include the possibility of third-party actors developing add-on functionality; a shorter time to market for new functions; and the ability to upgrade existing products in the field. This will however require not only new technical solutions, but also a transformation of the software ecosystems for ES. This paper aims at providing an initial characterization of the mechanisms that need to be present to make a FES ecosystem successful. This includes identification of the actors, the possible business models, the effects on product development processes, methods and tools, as well as on the product architecture. The research was carried out as an explorative case study based on interviews with 15 senior staff members at 9 companies related to ES that represent different roles in a future ecosystem for FES. The interview data was analyzed and the findings were mapped according to the Business Model Canvas (BMC). The findings from the study describe the main characteristics of a FES ecosystem, and identify the challenges for future research and practice. The case study indicates that new actors exist in the FES ecosystem compared to a traditional supply chain, and that their roles and relations are redefined. The business models include new revenue streams and services, but also create the need for trade-offs between, e.g., openness and dependability in the architecture, as well as new ways of working.

ABSTRACT The paper provides an in depth analysis of empirical evidence on the state of practice within the agile domain obtained through a survey conducted in 2012. The context of focus is agile software processes and teams and the particular topics of interest revolve around three axes: (i) communication; (ii) project management; and (iii) quality assurance and validation. The aim of the survey is to deliver the current levels of agile adoption and practices as these are recorded in the responses of professionals in IT services and the software industry. The goal of the survey is to provide evidence‐based assessment of the level of agile adoption by software development organizations, in relation to the general profile of the respondents (country of origin, business sectors, roles, etc.) and compared with different types of practices followed, such as agile techniques adopted, team organization and communication techniques, and project management. Particular patterns and trends are identified in the survey connecting the use of the agile paradigm with the aforementioned practices and investigating its relation with the roles of the respondents and the business strategies of their organizations. Copyright © 2014 John Wiley & Sons, Ltd.

Software cost estimation (SCE) is one of the critical activities in software project management. During the past decades various models have been proposed for SCE. However, developing accurate and useful models is limited in practice despite the considerable financial gain they could offer to software stakeholders. Traditional techniques, such as regression, by-analogy and machine learning, face the difficulty of handling the dynamic nature of the software process and the problematic nature of the public data available. This paper addresses the issue of SCE proposing an alternative approach that combines robust decision tree structures with fuzzy logic. Fuzzy decision trees are generated using the CHAID and CART algorithms in a systematic manner, while development effort is treated as the dependent variable against two subsets of factors: The first contains selected attributes from the ISBSG, COCOMO and DESHARNAIS datasets and the second contains a subset of the available factors that can be measured early in the development cycle. The association rules obtained from the trees are then merged and defuzzified through a Fuzzy Implication System (FIS). The fuzzy framework is utilized to perform effort estimations. Experimental results indicate that the proposed approach is promising as it yields quite accurate estimations in most dataset cases considered. Finally, our evaluation suggests that accurate estimations may be produced, even when using only a small set of factors that can be measured early in the development cycle, thus increasing the practical value of the proposed cost model.

This paper addresses the issue of software cost estimation through fuzzy decision trees, aiming at acquiring accurate and reliable effort estimates for project resource allocation and control. Two algorithms, namely CHAID and CART, are applied on empirical software cost data recorded in the ISBSG repository. Approximately 1000 project data records are selected for analysis and experimentation, with fuzzy decision trees instances being generated and evaluated based on prediction accuracy. The set of association rules extracted is used for providing mean effort value ranges. The experimental results suggest that the proposed approach may provide accurate cost predictions in terms of effort. In addition, there is strong evidence that the fuzzy transformation of cost drivers contribute to enhancing the estimation process.

The software cost estimation process is one of the most critical managerial activities related to project planning, resource allocation and control. As software development is a highly dynamic procedure, the difficulty of providing accurate cost estimations tends to increase with development complexity. The inherent problems of the estimation process stem from its dependence on several complex variables, whose values are often imprecise, unknown, or incomplete, and their interrelationships are not easy to comprehend. Current software cost estimation models do not inspire enough confidence and accuracy with their predictions. This is mainly due to the models' sensitivity to project data values, and this problem is amplified because of the vast variances found in historical project attribute data. This paper aspires to provide a framework for evolving value ranges for cost attributes and attaining mean effort values using the Al-oriented problem-solving approach of genetic algorithms, with a twofold aim. Firstly, to provide effort estimations by analogy to the projects classified in the evolved ranges and secondly, to identify any present correlations between effort and cost attributes.