Engineering intelligent systems for industrial edge device fleets is often hindered by the non-functional requirements of data privacy, transmission cost, and client heterogeneity, which make traditional centralized architectures impractical. This paper presents a generalizable architectural blueprint designed to address these constraints for resource-constrained, time-series forecasting tasks. The blueprint is founded on a dual-adaptation mechanism that combines a longterm, asynchronous Federated Learning (FL) protocol with a lightweight, on-device correction mechanism for rapid, real-time adaptation. To validate this blueprint, we apply it to the challenging use case of real-time State of Charge (SoC) prediction for a heterogeneous fleet of commercial Battery Electric Vehicles (BEVs) and the NASA C-MAPSS dataset to evaluate Remaining Useful life (RUL) of jetengines. The system was implemented and evaluated on production-intent embedded ARM hardware. The results demonstrate that the blueprint enables a privacypreserving federated system to achieve a final prediction error nearly on par with a non-private, fully centralized model. In the SoC experiment, the client with the least amount of data, reduced the median end-of-trip error from over 4% in a local-only model to under 1.5% while the RUL experiment reduced the local model median error of 5.85 cycles down to 0.16 using the blueprint. This work provides a definitive blueprint for deploying effective, privacy-preserving AI systems in industrial embedded systems, proving that a thoughtful engineering approach allows performance and privacy to coexist without compromise.

Software-intensive systems companies face mounting pressures to accelerate time-to-market. This drives adoption of DevOps, frequent deployments and AI-enabled analytics. However, while traditional feedback loops channel usage data, logs and interactions into development cycles, this doesn’t necessarily translate into learning. Although companies use DevOps, they still view product development as building products with a fixed scope. To address this, we conceptualize the notion of learning loops and how companies move towards continuous improvement of product performance. In our view,’learning loops’ distinguish themselves by translating data from products into actionable improvements executed by humans, by traditional ML, by Agentic AI or by a combination of these. This paper synthesizes longitudinal case study research and interviews, revealing that mechanisms like continuous integration and deployment, A/B testing, federated and reinforcement learning are unified instances of post-deployment learning loops. The contribution of this paper is two-fold. First, we provide empirical examples reflecting how R&D teams and systems learn and improve performance over time. Second, we present a conceptual model in which we detail the concept of learning loops that can be executed by humans, by traditional ML, by Agentic AI or by a combination of these.

Within the Engineering, Procurement, and Construction (EPC) industry, engineers manually create documents based on engineering drawings, which can be time-consuming and prone to human error. For example, the expansion of typical assemblies of instrument items (Instrument Typicals) in Piping and Instrumentation Diagrams (P&IDs) is a labor-intensive task. Each Instrument Typical assembly is depicted in the P&IDs via a simplified representation showing only a subset of the utilized instruments. The expansion activity involves recording all utilized instruments to create an instrument item list document based on the P&IDs for a particular EPC project. Fortunately, Artificial Intelligence (AI) could help to automate this process. In this paper, we propose the first method for automating the process of Instrument Typical expansion in P&IDs. The method utilizes computer vision techniques and domain knowledge rules to extract information about the Instrument Typicals from a project's P&IDs and legend sheets. Subsequently, the extracted information is used to automatically generate the listing of all utilized instruments. The effectiveness of our method is evaluated on P&IDs from large industrial EPC projects, resulting in precision rates exceeding 98% and recall rates surpassing 99%. These results demonstrate the suitability of our method for industrial deployment. The successful application of our method has the potential to reduce engineering costs and increase the efficiency of EPC projects. Furthermore, the method could be adapted for additional applications in the EPC industry, which highlights the method's industrial value.

Platform ecosystems have revolutionized value creation across numerous industries, inducing technical leaps and scaling artifact generation in scales and speeds unattainable through traditional vertically integrated or single-firm models. These ecosystems rely on collaborative interactions among actors to co-create and reuse value. Such ecosystems are socio-technical environments which require complex governance and orchestration strategies to ensure ecosystem health and performance from financial, technical, and social perspectives. Consequently, monitoring the performance of platform ecosystems requires non-primitive metrics as factors contributing to ecosystem performance are multifaceted compared to conventional software settings. Effective orchestration of platform ecosystems requires relies on access to real-time quantitative performance indicators. The existing literature offers various quantitative health metrics and performance indicators for platform ecosystems, yet these are dispersed across multiple studies and often embedded in abstract models or found within generic analytics systems. This research reviews existing quantitative real-time health metrics and performance indicators of platform ecosystems. We identified 417 distinct metrics after eliminating duplicates and incomplete definitions, and refined 168 of these metrics to be calculable in real-time using a consistent framework for definition, nomenclature, and quantification. Furthermore, we compiled existing real-time ecosystem health monitoring methods into a reference architecture and tested its feasibility in two active platform ecosystems. The study yields four key contributions: a practical catalog of platform ecosystem health metrics; a reference architecture for creating real-time ecosystem health monitoring solutions, demonstrated through implementation in two operational platform ecosystems; industry-relevant insights for practitioners; and a discussion of potential future research directions.

Engineering, Procurement, and Construction (EPC) projects often encounter critical inefficiencies in Interdisciplinary Check (IDC) processes, where engineers manually validate design consistency across drawings and specifications, resulting in bottlenecks in large-scale projects. This paper introduces a proof-of-concept(PoC) human-AI cooperative framework for automating IDC workflows. The framework combines deep learning for document processing, heuristic algorithms for feature extraction, and a Large Multimodal Model (LMM) for cross-domain validation, while preserving engineering decision authority. The framework extracts key engineering data, links components across disciplines, and checks compliance with industry standards and client requirements. Testing on controlled data from a single EPC project (with 25 different engineering drawings) successfully identified all design discrepancies within the predefined validation scope, demonstrating notable workflow efficiency potential. As Phase 1 of a three-phase research cycle, this confirms technical feasibility before expanding to multiple projects and company-wide deployment, showing how human-AI collaboration can revolutionize engineering validation while preserving oversight of engineering expertise.

Interpreting complex engineering drawings requires substantial manual effort in industrial workflows, with engineers spending hundreds of hours verifying correlations between visual elements and structured specifications across thousands of documents. This challenge is particularly acute in the Engineering, Procurement and Construction (EPC) industries, where interpretation errors are propagated into costly procurement and construction mistakes. We propose a cybernetic systems approach using Large Multimodal Models (LMMs) as cognitive partners in engineering documentation workflows, enhancing human capabilities through intelligent assistance in verification and information extraction tasks. To validate this solution, we systematically evaluated five LMMs on 60 piping isometric drawings with varying template structures, measuring both optical character recognition accuracy and correlation capabilities between drawings and their associated Bills of Materials. The results demonstrated significant performance variation between systems, with Claude 3.5 Sonnet and GPT-4o achieving greater accuracy 70%, while open source alternatives faced challenges with complex layouts and ambiguous visual information. These findings establish benchmarks for humanmachine collaboration in engineering documentation processing and provide a framework for integrating intelligent systems into technical workflows across multiple industrial domains.

Context: Machine Learning Operations (MLOps) has become a top priority for companies. However, its adoption has become challenging due to the need for proper guidance and awareness. Most of the MLOps solutions available in the market are designed to fit the specific platform, tools and culture of the providers. Objective: The objective is to develop a structured approach to adopting, assessing and advancing MLOps adoption. Methods: The study was conducted based on a multi-case study across fourteen companies. Results: We provide a comprehensive analysis that highlights the similarities and differences in the adoption of MLOps practices among companies. We have also empirically validated the developed MLOps framework and MLOps maturity model. Furthermore, we carefully reviewed the feedback received from practitioners and revised the MLOps framework and maturity model to confirm its effectiveness. Additionally, we develop an MLOps taxonomy for classifying ML use cases based on their context and requirements into the desired stage of the MLOps framework and maturity model. Conclusion: The findings provide companies with a structured approach to adopt, assess, and further advance the adoption of MLOps practices regardless of their current status.

Context: The role of product management (PM) is key for building, implementing and managing software-intensive systems. Whereas engineering is concerned with how to build systems, PM is concerned with ‘what’ to build and ‘why’ we should build the product. The role of PM is recognized as critical for the success of any product. However, few studies explore how the role of PM is changing due to recent trends that come with digitalization and digital transformation. Objectives: Although there is prominent research on PM, few studies explore how this role is changing due to the digital transformation of the software-intensive industry. In this paper, we study how trends such as DevOps and short feedback loops, data and artificial intelligence (AI), as well as the emergence of digital ecosystems, are changing current product management practices. Methods: This study employs a qualitative approach using multi-case study research as the method. For our research, we selected five case companies in the software-intensive systems domain. Through workshop sessions, frequent meetings and interviews, we explore how DevOps and short feedback loops, data and artificial intelligence (AI), and digital ecosystems challenge current PM practices. Results: Our study yielded an in-depth understanding of how digital transformation of the software-intensive systems industry is changing current PM practices. We present empirical results from workshops and from interviews in which case company representatives share their insights on how software, data and AI impact current PM practices. Based on these results, we present a framework organized along two dimensions, i.e. a certainty dimension and an approach dimension. The framework helps structure the approaches product managers can employ to select and prioritize development of new functionality. Contributions: The contribution of this paper is a framework for ‘Strategic Digital Product Management’ (SDPM). The framework outlines nine approaches that product managers can employ to maximize the return on investment (RoI) of R&D using new digital technologies.

Federated Learning (FL) has gained prominence as a solution for preserving data privacy in machine learning applications. However, existing FL frameworks pose challenges for software engineers due to implementation complexity, limited customization options, and scalability issues. These limitations prevent the practical deployment of FL, especially in dynamic and resource-constrained edge environments, preventing its widespread adoption. To address these challenges, we propose EdgeFL, an efficient and low-effort FL framework designed to overcome centralized aggregation, implementation complexity and scalability limitations. EdgeFL applies a decentralized architecture that eliminates reliance on a central server by enabling direct model training and aggregation among edge nodes, which enhances fault tolerance and adaptability to diverse edge environments. We conducted experiments and a case study to demonstrate the effectiveness of EdgeFL. Our approach focuses on reducing weight update latency and facilitating faster model evolution on edge devices. Our findings indicate that EdgeFL outperforms existing FL frameworks in terms of learning efficiency and performance. By enabling quicker model evolution on edge devices, EdgeFL enhances overall efficiency and responsiveness to changing data patterns. EdgeFL offers a solution for software engineers and companies seeking the benefits of FL, while effectively overcoming the challenges and privacy concerns associated with traditional FL frameworks. Its decentralized approach, simplified implementation, combined with enhanced customization and fault tolerance, make it suitable for diverse applications and industries.

ABSTRACT Process debt, like technical debt, can be a source of short‐term benefits but often leads to harmful consequences in the long term for a software organization. Despite its impact, the phenomenon of process debt has not been thoroughly explored in current literature, leaving a gap in understanding how it affects and is managed within organizations. This paper addresses this gap by defining process debt, describing its occurrence, the risks of its mismanagement, and showing examples of mitigation strategies. Our study began with an exploratory phase involving semi‐structured interviews with sixteen practitioners across four international organizations, allowing us to gather diverse insights into the occurrence and management of process debt. Then, to deepen our understanding and validate our findings, we conducted a cross‐company focus group with ten additional practitioners and analyzed fifty‐eight observations and thirty‐five interviews from a longitudinal case study. The analysis of the research findings led to a definition of process debt and a novel framework. We also report on the causes, consequences, and occurrence patterns of process debt over time. We present mitigation strategies and discuss which ones need further attention for future research. Our results suggest that the debt metaphor may help companies understand how to manage and improve their processes and make process‐related decisions that are beneficial both in the short and long term.

ABSTRACT Manual analysis of diagrams and legend sheets in engineering projects is time consuming and needs automation. The lack of standardized legend formats complicates creating a general method for automated information extraction. Existing approaches require training and custom rules for each project. This study proposes a novel solution combining optical character recognition with vision language models and multimodal prompt engineering to automate information extraction from diverse legend sheets without training. It integrates legend information with information extracted from diagrams, unlike studies that only focus on diagrams. Our study shows that VLMs, guided by multimodal prompts, can accurately extract information from diverse legend sheets, enabling automatic information extraction in diagrams across engineering projects. We validate our method through a case study involving the extraction of instruments from piping and instrumentation diagrams (P&IDs) and their legends across three projects with varied formats and standards. The proposed method achieved 100% accuracy in legend classification and information extraction, and 99.68% precision and 95.91% recall in generating instrument listings. The results demonstrate the effectiveness of our approach, significantly enhancing the accuracy and efficiency of information extraction from diagrams. This method can be adapted to different legend formats and diagrams, providing a versatile solution for various industries.

Continuous deployment aims to reduce the deployment cycle of new software, enabling software development companies to deliver new and improved functionalities to customers faster and more frequently. In software-intensive embedded systems, continuous deployment is often introduced as a subsequent step to continuous integration. While several empirical studies explored the transition from continuous integration to continuous deployment from several aspects, such as challenges, benefits, and success factors, these studies remain limited as they focus on applying continuous deployment to a small subset of the entire customer base. However, as software-intensive embedded systems are often high-volume products used and operated by many different customers, scaling continuous deployment becomes important to ensure that all customers perceive its benefits while at the same time enabling the software development organization to have one release and deployment cycle applicable to the entire customer base. Thus, to further understand the progression of continuous deployment from introduction to scaling, we conducted a longitudinal case study at a multinational telecommunications company producing complex telecommunications software-intensive embedded systems. Our results show that continuous deployment passes through four phases: R&D experiment, R&D core practice, as a service, and finally, continuous deployment supporting a result-oriented business model. Based on the results, we inductively derive a maturity model for continuous deployment, which we discuss based on the four dimensions of the BAPO framework (Business, Architecture, Process, and Organization).

Federated Learning (FL) has emerged as a promising approach for collaborative machine learning, addressing data privacy concerns. As data security and privacy concerns continue to gain prominence, FL stands out as an option to enable organizations to leverage collective knowledge without compromising sensitive data. However, existing FL platforms and frameworks often present challenges for software engineers in terms of complexity, limited customization options, and scalability limitations. In this paper, we introduce EdgeFL, an edge-only lightweight decentralized FL framework, designed to overcome the limitations of centralized aggregation and scalability in FL deployments. By adopting an edge-only model training and aggregation approach, EdgeFL eliminates the need for a central server, enabling seamless scalability across diverse use cases. Our results show that EdgeFL reduces weights update latency and enables faster model evolution, enhancing the efficiency of edge model learning. Moreover, EdgeFL exhibits improved classification accuracy compared to traditional centralized FL approaches. By leveraging EdgeFL, software engineers can harness the benefits of Federated Learning while overcoming the challenges associated with existing FL platforms/frameworks.

DevOps has become a widely adopted approach in the software industry, especially among companies developing web-based applications. The main focus of DevOps is to address social and technical bottlenecks along the software flow, from the developers' code changes to delivering these changes to the production environments used by customers. However, DevOps does not consider the software flow's content, e.g., new features, bug fixes, or security patches, and the customer value of each content. In addition, DevOps assumes that a streamlined software flow leads to a continuous value flow, as customers use the new software and extract value-adding content intuitively. However, in a Software-intensive System of Systems (SiSoS), customers need to understand the content of the software flow to validate, test, and adopt their operation procedures before using the new software. Thus, while DevOps has been extensively studied in the context of web-based applications, its adoption in SiSoS is a relatively unexplored area. Therefore, we conducted a case study at a multinational telecommunications provider focusing on 5G systems. Our findings reveal that DevOps has three sub-flows: legacy, feature, and solution. Each sub-flow has distinct content and customer value, requiring a unique approach to extracting it. Our findings highlight the importance of understanding the software flow's content and how each content's value can be extracted when adopting DevOps in SiSoS.

Context: Software ecosystems have transformed many industries, redefining collaboration and value co-creation. The success of such ecosystems depends on the dynamism of the network of users on its different sides. Consequently, decision-making in such multifaceted and interconnected environments is more complex than in conventional software products. On-line controlled experiments are considered the gold standard for aiding decision-making in software engineering processes. Experiments are extensively used to reduce bias and estimation noise for design, engineering, and business decisions. However, experimentation in software ecosystems is inherently more com-plex as it deals with atypical sources of bias and technical complications. Primary studies of experimentation approaches in software ecosystems are scattered across multiple domains and disciplines, and secondary research on the topic is scarce as highlighted in different tertiary studies. Hence, we conducted this study. Objectives: To explore primary research on experimentation in software ecosystems; Summarize current approaches, toolboxes, and solutions that practitioners and researchers, facing similar problems, can use to inform their approaches; To outline underexplored research areas and provide recommendations for practitioners. Method: We conducted a systematic literature review. The search strategy, application of exclusion and inclusion criteria, and subsequent quality assessment resulted in 63 relevant studies. Data extraction process was designed and carried out to collect data relevant to the study objectives. The extracted data under-went descriptive and thematic syntheses and analyses, in addition to cross-analysis on relevant axes. Contributions: The study resulted in four contributions. First, a distillation of the themes and patterns in the available research on the topic. Second, a practical summary of the experimental designs specific to each software ecosystem type. Third, an actionable road map for practitioners in order to achieve exper-imentation maturity in software ecosystems. Fourth, an outline of the underexplored research areas.

Industrial software ecosystems refer to a network of interdependent actors, co-creating value through a shared technological platform specifically tailored to industrial sectors. Developing, maintaining, and orchestrating such platforms involves many challenges that require complex decision making. Experimentation can help alleviate this complexity and reduce decision uncertainty and bias. However, experimentation requires certain organizational, infrastructural, and data-related prerequisites which can be uniquely challenging to achieve in industrial software ecosystems. Through semi-structured interviews with 25 industry professionals involved in various roles across 17 ecosystems, we analyze the difficulties faced in conducting effective experiments in such environments. The interview protocol covered aspects related to the methodologies, data handling processes, and current experimentation practices, as well as the challenges faced by practitioners who engage in experimentation initiatives. The study findings reveal technical, organizational, and market-related challenges, detailing the complexities facing experimentation initiatives in industrial software ecosystems. The findings are presented in an actionable manner, following a model that allows business-oriented alignment of architecture, process, and organizational evolution strategies. The study identifies key impediments, such as data integration difficulties, stringent regulatory environments, and prevailing organizational cultures that hinder continuous experimentation practices. Our analysis provides a foundation for understanding the unique challenges facing experimentation efforts in industrial software ecosystems and offers insights into potential strategies to improve the effectiveness of these initiatives.

Data is key for rapid and continuous delivery of customer value. By collecting data from products in the field, companies in the embedded systems domain can measure and monitor product performance and they get the opportunity to provide customers with insights and data-driven services. However, while the notion of data-driven development is not new, embedded systems companies are facing a situation in which data volumes are growing exponentially and this is not without its challenges. Suddenly, the cost of collecting, storing and processing data becomes a concern and while there is prominent research on different aspects of data-driven development, there is little guidance for how to reason about business value versus costs of data. In this paper, we present findings from case study research conducted in close collaboration with four companies in the embedded systems domain. The contribution of this paper is a framework that provides a holistic understanding of the multiple dimensions that need to be considered when reasoning about business value versus cost of collecting, storing and processing data.

The success of any Engineering, Procurement, and Construction (EPC) project depends on the engineering deliverables developed during project execution. An important deliverable is the Line List document, produced by extracting pipeline numbers from Piping and Instrumentation Diagrams (P&IDs). As the creation of this document is time-consuming, the automation of this process could reduce manual engineering work. However, the complexity of the P&IDs renders traditional computer vision approaches unsuitable. Therefore, deep learning text detection could be utilized to achieve this task. This study assessed the applicability of text detection methods for automating pipeline number information extraction in P&IDs. Our findings indicate that the methods previously used to detect text on P&IDs have limitations in accurately capturing the entire line numbers. Furthermore, we propose a line number detection method achieving a recall rate of over 90% on our evaluation data, consisting of P&IDs from diverse industrial projects. Thus, we demonstrate our method's generalizability to different line number formats and its potential for industrial application. Moreover, the proposed method can be adapted to other types of engineering drawings beyond P&IDs. Thus, it could be used in additional applications for digitizing engineering drawings.

Applied Artificial Intelligence (AI) in engineering is gaining significant traction. AI object detection methods can be applied in the engineering industry to extract information from engineering drawings, offering immense benefits to engineers. A promising application of AI in industrial engineering is symbol recognition applied to engineering drawings. However, these drawings often exhibit areas with a high density of symbols, as well as noise in the form of markups, indicating revisions. These factors could cause symbol misclassification or omission, impacting applications reliant on accurate symbol recognition. This study evaluates the accuracy of a symbol recognition model on engineering drawings called Piping and Instrumen-tation Diagrams (P &IDs) exhibiting varying levels of density and markups causing noise. Despite the assumption that density poses a challenge for accurate symbol recognition in engineering drawings, our study reveals that density has no significant impact on recognition performance when a dense detector is employed. In addition, we quantitatively show that markup-induced noise on engineering drawings negatively influences recognition accuracy. Finally, we provide recommendations regarding the applicability of symbol recognition in engineering applications. The study's findings and recommendations apply to any P &IDs, regardless of the standard used, as they were evaluated on various worldwide projects. Moreover, the research not only contributes to the advancement of symbol recognition on P&IDs, but also can be applied to other types of engineering drawings. Thus, it holds the potential for enhancing symbol recognition in various real-world industrial applications and research.

In the Engineering, Procurement, and Construction (EPC) sector, accurate cost estimations during the tendering phase are crucial for maintaining competitiveness, especially with constrained project schedules and rising labor expenses. Typically, these estimations are labor-intensive, relying heavily on manual evaluations of engineering drawings, which are often shared in PDF format due to intellectual property concerns. This study introduces an innovative solution tailored for the energy industry, utilizing Artificial Intelligence (AI) - primarily deep learning (DL) and machine learning (ML) techniques - to streamline material quantity estimation, thereby saving engineering time and costs. Built on empirical data from a large EPC company operating in the energy sector, AI-based product development experiences, and academic research, our approach aims to enhance the efficiency and accuracy of engineering work, promoting better decision-making and resource distribution. While our focus is on enhancing a particular activity within the case company using AI, the method's broader applicability in the EPC sector potentially benefits both industry professionals and researchers. This study not only advances a practical application but also provides valuable insights for those seeking to develop AI -driven solutions across various engineering disciplines.

The Engineering, Procurement and Construction (EPC) businesses operating within the energy sector are recognizing the increasing importance of Artificial Intelligence (AI). Many EPC companies and their clients have realized the benefits of applying AI to their businesses in order to reduce manual work, drive productivity, and streamline future operations of engineered installations in a highly competitive industry. The current AI market offers various solutions and services to support this industry, but organizations must understand how to acquire AI technology in the most beneficial way based on their business strategy and available resources. This paper presents a framework for EPC companies in their transformation towards AI. Our work is based on examples of project execution of AI-based products development at one of the biggest EPC contractors worldwide and on insights from EPC vendor companies already integrating AI into their engineering solutions. The paper covers the entire life cycle of building AI solutions, from initial business understanding to deployment and further evolution. The framework identifies how various factors influence the choice of approach toward AI project development within large international engineering corporations. By presenting a practical guide for optimal approach selection, this paper contributes to the research in AI project management and organizational strategies for integrating AI technology into businesses. The framework might also help engineering companies choose the optimum AI approach to create business value.

Abstract Continuous deployment has been practiced for many years by companies developing web‐ and cloud‐based applications. To succeed with continuous deployment, these companies have a strong collaboration culture between the operations and development teams. In addition, these companies use AI, analytics, and big data to assist with time‐consuming postdeployment activities such as continuous monitoring and fault identification. Thus, the term AIOps has evolved to highlight the importance and difficulty of maintaining highly available applications in a complex and dynamic environment. In contrast, software‐intensive embedded systems often provide customer product‐related services, such as maintenance, optimization, and support. These services are critical for these companies as they provide significant revenue and increase customer satisfaction. Therefore, the objective of our study is to gain an in‐depth understanding of the impact of continuous deployment on product‐related services provided by software‐intensive embedded systems companies. In addition, we aim to understand how AIOps can support continuous deployment in the context of software‐intensive embedded systems. To address this objective, we conducted a case study at a large and multinational telecommunications systems provider focusing on the radio access network (RAN) systems for 4G and 5G networks. The company provides RAN products and three complementing services: rollout, optimization, and customer support. The results from the case study show that the boundaries between product‐related services become blurry with continuous deployment. In addition, product‐related services, which were conducted in sequence by independent projects, converge with continuous deployment and become part of the same project. Further, AIOps platforms play an important role in reducing costs and increasing postdeployment activities' efficiency and speed. These results show that continuous deployment has a profound impact on the software‐intensive system's provider service organization. The service organization becomes the connection between the R&D organization and the customer. In order to cope with the increased speed of releases, deployment and postdeployment activities need to be largely automated. AIOps platforms are seen as a critical enabler in managing the increasing complexity without increasing human involvement.

Machine Learning Operations (MLOps) play a crucial role in the success of data science projects in companies. However, despite its obvious benefits, several companies struggle to adopt MLOps practices and face difficulty in deciding how to deploy and evolve ML models. To gain a deeper understanding of these challenges, we conduct a multi-case study involving nine practitioners from seven companies. Based on our empirical results, we identify the key trade-offs we see companies make when adopting MLOps. We categorise these trade-offs into four concerns of the BAPO model: Business, Architecture, Process, and Organisation. Finally, we provide suggestions to mitigate the identified trade-offs. By identifying and detailing these trade-offs and the implications of these, this research helps companies to ensure the successful adoption of MLOps.

Data pipelines can be defined as a complex chain of interconnected activities that starts with a data source and ends in a data sink. They can process data in multiple formats from various data sources with minimal human intervention, speed up data life cycle operations, and enhance productivity in data-driven organizations. As a result, companies place a high value on strengthening the maturity of their data pipelines. The available literature, on the other hand, is significantly insufficient in terms of providing a comprehensive roadmap to guide companies in assessing the maturity of their data pipelines. Therefore, this case study focuses on developing a data pipeline maturity assessment model that can evaluate the maturity of data pipelines in a staged manner from maturity level 1 to maturity level 5. We conducted empirical research in order to develop the maturity assessment model on the basis of five different determinants to address the specific needs of each data pipeline maturity level. Accordingly, it aims to support organizations in assessing their current data pipeline maturity, determining challenges at each stage, and preparing an extensive roadmap and suggestions for data pipeline maturity improvement. In future work, we plan to employ the maturity model in different companies as a case study to evaluate its applicability and usefulness.

Deep reinforcement learning has advanced signifi-cantly in recent years, and it is now used in embedded systems in addition to simulators and games. Reinforcement Learning (RL) algorithms are currently being used to enhance device operation so that they can learn on their own and offer clients better services. It has recently been studied in a variety of industrial applications. However, reinforcement learning, especially when controlling a large number of agents in an industrial environment, has been demonstrated to be unstable and unable to adapt to realistic situations when used in a real-world setting. To address this problem, the goal of this study is to enable multiple reinforcement learning agents to independently learn control policies on their own in dynamic industrial contexts. In order to solve the problem, we propose a dynamic multi-agent reinforcement learning (dynamic multi-RL) method along with adaptive exploration (AE) and vector-based action selection (VAS) techniques for accelerating model convergence and adapting to a complex industrial environment. The proposed algorithm is tested for validation in emergency situations within the telecommunications industry. In such circumstances, three unmanned aerial vehicles (UAV-BSs) are used to provide temporary coverage to mission-critical (MC) customers in disaster zones when the original serving base station (BS) is destroyed by natural disasters. The algorithm directs the participating agents automatically to enhance service quality. Our findings demonstrate that the proposed dynamic multi-RL algorithm can proficiently manage the learning of multiple agents and adjust to dynamic industrial environments. Additionally, it enhances learning speed and improves the quality of service.

Companies producing software-intensive products do not only offer products to customers but Product Service Systems (PSS), a combination of the products and services that address customers’ needs. Further, product-related services are key in ensuring customer satisfaction as the service organization represents the company’s interface toward its customers, who operate and use the products. Therefore, while DevOps has been widely adopted in companies developing web-based applications aiming to streamline the Development and Operations activities, the projecting of DevOps as applied in web-based applications to PSS is difficult without considering the role of services. Therefore, based on a two years participant observation case study conducted at a multinational telecommunications systems provider, we propose a new and novel approach called Development-Services-Operations (DevServOps) which incorporates services as a key player facilitating an end-to-end software flow toward customers in one direction and feedback toward developers in the other direction.

In recent years, the interest in applying machine learning (ML) and deep learning (DL) has been increasing due to their ability to learn to predict and find structure in data. The most common approach of ML and DL is supervised learning. Supervised learning requires the input data to be labeled. However, as reported by many industries, such as the embedded systems domain, fully labeled datasets are difficult to obtain since data labeling is manually intensive. This paper uses a semi-supervised learning approach on real-world Pulse-Doppler data obtained from our industry collaborator Saab to address this challenge. We took inspiration from the FixMatch algorithm. To investigate whether unlabeled data can help improve classification accuracy, we compare FixMatch to a supervised baseline. We use five different settings for the number of available labels per class label to investigate how many labeled instances and how much manual effort is required for optimal accuracy. Bayesian Linear Regression is used to analyze the results. The results show that FixMatch can reach a higher accuracy than the supervised baseline. Furthermore, FixMatch requires more computation time but will help reduce manual effort. In addition, FixMatch will not underfit or overfit. Thanks to this study, practitioners know the benefits of utilizing FixMatch and when it is safe to use to improve a supervised baseline in the industry.

The emergence of platform ecosystems has transformed the business landscape in many industries, giving rise to novel modes of interorganizational cooperation and value co-creation, as well as unconventional challenges. The vast traces of data generated by platform ecosystems makes them ripe for the use of analytics and data-driven methods aimed at improving their health, performance, business outcomes, and evolution. However, the research on the application of analytics within platform ecosystems is limited and spread across multiple disciplines. To address this gap, we conducted a systematic literature review on the application of analytics and data-driven methods and practices within platform ecosystems. A total of 56 studies were reviewed, and underwent data extraction, analysis, and synthesis processes. In addition to presenting themes and patterns in the recent and relevant literature on platform ecosystems analytics, our review offers the following outcomes: an actionable overview of the analytics toolbox currently used within platform ecosystems—spanning domains such as machine learning, deep learning, data science, modelling, simulation, among others—; a roadmap for practitioners to achieve analytics maturity; and a summary of underexplored research areas.

As industries continue to digitize at a rapid pace, the number of integrations, the connections between different digital systems, is increasing. As a consequence, integration management is becoming increasingly difficult, leading to scalability, stability, and system outage problems that directly affect company performance. For that reason the modern integration management platforms have emerged to offer efficient management of the growing number of integrations, as well as a new way to handle, enrich, and utilize data both inside and outside organizations.This research involved interviews with 20 software vendor professionals and 20 clients who had recently undergone an integration platform project. The study identified key drivers behind integration management projects and recognized different maturity levels in integration management. The results revealed that more mature companies are utilizing modern integration platforms to support their digital transformation. The study also highlighted the growing importance of data management and utilization in integration solutions.To address these findings, a maturity model was developed to help companies evaluate their current integration management status and find ways to progress in their integration management.

Companies across various domains increasingly adopt Machine Learning Operations (MLOps) as they recognise the significance of operationalising ML models. Despite growing interest from practitioners and ongoing research, MLOps adoption in practice is still in its initial stages. To explore the adoption of MLOps, we employ a multi-case study in seven companies. Based on empirical findings, we propose a maturity model outlining the typical stages companies undergo when adopting MLOps, ranging from Ad hoc to Kaizen. We identify five dimensions associated with each stage of the maturity model as part of our MLOps framework. We also map these seven companies to the identified stages in the maturity model. Our study serves as a roadmap for companies to assess their current state of MLOps, identify gaps and overcome obstacles to successfully adopting MLOps.

Effective collection and use of data is key for companies across domains and it is only increasing in importance. For companies in the embedded systems domain, data constitutes the basis not only for quality assurance and diagnostics of their systems but also for new service development and innovation. For these companies, data is an enabler for continuous delivery of customer value and hence, a key asset for entirely new and recurring revenue streams. However, effective use of data requires careful collection of different kinds of data depending on the purpose and context for which it is intended to be used. In this paper, we identify the challenges that companies experience in their contemporary data practices and we outline the kinds of data that companies need to collect as they evolve through different maturity stages. In addition, we provide concrete guidance on the specific data to collect during each maturity stage.

In recent years, Federated Learning, as an approach to distributed learning, has shown its potential with the increasing number of devices on the edge and the development of computing power. The method enables large-scale training on the device that creates the data but with the sensitive data remaining within the data’s owner. In reality, however, the vast majority of enterprises have the problem of low data volume and poor model quality to support the implementation of Federated Learning methods. Learning quality assurance for edge devices is still the major issue which prevents Federated Learning to be applied in industrial contexts, especially in safety-critical applications. In this paper, we propose a quality-based asynchronous Federated Learning algorithm (QuaFedAsync) to address these challenges. We report on a study in which we used two well-known data sets, i.e., DDAD and KITTI datasets, and validate the proposed algorithm on an industrial use case concerned with monocular depth estimation in the automotive domain. Our results show that the proposed algorithm significantly improves the prediction performance compared to the commonly applied aggregation protocols while maintaining the same level of accuracy as centralized machine learning. Based on the results, we prove the learning efficiency and robustness when applying the algorithm to industrial scenarios.

Abstract Artificial intelligence is increasingly becoming important to businesses because many companies have realized the benefits of applying machine learning (ML) and deep learning (DL) in their operations. ML and DL have become attractive technologies for organizations looking to automate repetitive tasks to reduce manual work and free up resources for innovation. Unlike rule‐based automation, typically used for standardized and predictable processes, machine learning, especially deep learning, can handle more complex tasks and learn over time, leading to greater accuracy and efficiency improvements. One of such promising applications is to use AI to reduce manual engineering work. This paper discusses a particular case within McDermott where the research team developed a DL model to do a quality check of complex blueprints. We describe the development and the final product of this case—AI‐based software for the engineering, procurement, and construction (EPC) industry that helps to find the design mistakes buried inside very complex engineering drawings called piping and instrumentation diagrams (P&IDs). We also present a cost‐benefit analysis and potential scale‐up of the developed software. Our goal is to share the successful experience of AI‐based product development that can substantially reduce the engineering hours and, therefore, reduce the project's overall costs. The developed solution can also be potentially applied to other EPC companies doing a similar design for complex installations with high safety standards like oil and gas or petrochemical plants because the design errors it captures are common within this industry. It also could motivate practitioners and researchers to create similar products for the various fields within engineering industry.

Abstract Artificial intelligence (AI) and the use of machine learning (ML) and deep learning (DL) technologies are becoming increasingly popular in companies. These technologies enable companies to leverage big quantities of data to improve system performance and accelerate business development. However, despite the appeal of ML/DL, there is a lack of systematic and structured methods and processes to help data scientists and other company roles and functions to develop, deploy and evolve models. In this paper, based on multi‐case study research in six companies, we explore practices and challenges practitioners experience in developing ML/DL models as part of large software‐intensive embedded systems. Based on our empirical findings, we derive a conceptual framework in which we identify three high‐level activities that companies perform in parallel with the development, deployment and evolution of models. Within this framework, we outline activities, iterations and triggers that optimize model design as well as roles and company functions. In this way, we provide practitioners with a blueprint for effectively integrating ML/DL model development into the business to achieve better results than other (algorithmic) approaches. In addition, we show how this framework helps companies solve the challenges we have identified and discuss checkpoints for terminating the business case.

Abstract Continuous experimentation (CE) refers to a set of practices used by software companies to rapidly assess the usage, value, and performance of deployed software using data collected from customers and systems in the field using an experimental methodology. However, despite its increasing popularity in developing web‐facing applications, CE has not been studied in the development process of business‐to‐business (B2B) mission‐critical systems. By observing the CE practices of different teams, with a case study methodology inside Ericsson, we were able to identify the different practices and techniques used in B2B mission‐critical systems and a description and classification of the four possible types of experiments. We present and analyze each of the four types of experiments with examples in the context of the mission‐critical long‐term evolution (4G) product. These examples show the general experimentation process followed by the teams and the use of the different CE practices and techniques. Based on these examples and the empirical data, we derived the HURRIER process to deliver high‐quality solutions that the customers value. Finally, we discuss the challenges, opportunities, and lessons learned from applying CE and the HURRIER process in B2B mission‐critical systems.

Artificial intelligence is increasingly becoming important to businesses since many companies have realized the benefits of applying Machine Learning (ML) and Deep Learning (DL) into their operations. Nevertheless, ML/DL technologies' industrial development and deployment examples are still rare and generally confined within a small cluster of large international companies who are struggling to apply ML more broadly and deploy their use cases at a large scale. Meanwhile, current AI market has started offering various solutions and services. Thus, organizations must understand how to acquire AI technology based on their business strategy and available resources. This paper discusses the industrial experience of developing and deploying ML/DL use cases to support organizations in their transformation towards AI. We identify how various factors, like cost, schedule, and intellectual property, can be affected by the choice of approach towards ML/DL project development and deployment within large international engineering corporations. As a research result, we present a framework that covers the trade-offs between those various factors and can support engineering companies to choose the best approach based on their long-term business strategies and, therefore, would help to accomplish their ML/DL project deployment successfully.

Supporting customers after they acquire the prod-uct is essential for companies producing and selling software-intensive embedded systems products. Generally, customer sup-port is the first interaction point between the product users and the product vendor. Customer support is often engaged with answering customers' questions, troubleshooting, fault identification, and fixing product faults. While continuous deployment advocates for closer cooperation between the ones operating the software and the ones developing it, the means of such collaboration in general and the role of customer support, in particular, has not been addressed in the context of software-intensive embedded systems. Therefore, to better understand the impact that continuous deployment has on customer support and the role customer support should play in this context, we conducted a case study at a multinational company developing and selling telecommunications networks infrastructure. We focused on the 4th and 5th Generation (4G and 5G) Radio Access Networks (RAN) products, which can be considered a high volume product as they cover more than 80% of the world's population. Our study reveals that customer support needs to transition from a transaction-based and passive function triggered by customer support requests, to take an active role characterized by being proactive and preemptive to cope with the shorter operational time of a software version introduced by continuous deployment. In addition, customer support plays an essential role in making the feedback actionable by aggregating and consolidating feedback data to the R&D organization.

AI in the Engineering, Procurement and Construction (EPC) industry has not yet a proven track record in large-scale projects. Since AI solutions for industrial applications became available only recently, deployment experience and lessons learned are still to be built up. Several research papers exist describing the potential of AI, and many surveys and white papers have been published indicating the challenges of AI deployment in the EPC industry. However, there is a recognizable shortage of in-depth studies of deployment experience in academic literature, particularly those focusing on the experiences of EPC companies involved in large-scale project execution with high safety standards, such as the petrochemical or energy sector. The novelty of this research is that we explore in detail the challenges and obstacles faced in developing and deploying AI in a large-scale project in the EPC industry based on real-life use cases performed in an EPC company. Those identified challenges are not linked to specific technology or a company's know-how and, therefore, are universal. The findings in this paper aim to provide feedback to academia to reduce the gap between research and practice experience. They also help reveal the hidden stones when implementing AI solutions in the industry.

Modern release engineering practices such as continuous integration and delivery have allowed software development companies to transition from a long release cycle to a shorter one. The shorter release cycle has led to more software releases available to customers. At the same time, companies developing high-volume software-intensive embedded systems often deliver patch releases and maintenance releases on top of major and minor releases to customers who pick and choose what releases apply to them and decide when to upgrade the system, if to upgrade at all. While release engineering has been studied before in web-based, desktop-based, and embedded software, the focus has been on pre-release activities. Few studies have investigated what happens after the release, particularly the role of tracing software from release to deployment in high-volume software-intensive embedded systems. To address this gap, we conducted a qualitative case study at a multi-national telecommunications systems provider focusing on Radio Access Network (RAN) software. RAN software is a complex and large-scale embedded software used in mobile networks Base Stations (BS), providing software functionality for RAN mobile technologies ranging from 2G to 5G. Our study shed light on post-release software traceability and how it is used in the release engineering process.

For companies in the embedded systems domain, digitalization and digital technologies allow endless opportunities for new business models and continuous value delivery. While physical products still provide the core revenue, these are rapidly being complemented with offerings that allow for recurring revenue and that are based on software, data and artificial intelligence (AI). However, while new digital offerings allow for fundamentally new and recurring revenue streams and continuous value delivery to customers, the creation of these proves to be a challenging endeavour. In this paper, we study how companies explore ways to create new or additional value with the intention to complement their product portfolio with offerings that allow for recurring revenue. Based on multi-case study research, we identify the key challenges that companies in the embedded systems domain experience and we derive four organizational patterns that we see slow down innovation. Second, we present a framework outlining alternative types of offerings to customers. Third, we provide a value taxonomy in which we detail the different types of offerings and the value these provide to customers. For each value offering, we indicate whether this offering is (1) static or evolving, (2) bundled or unbundled, (3) free or monetized, and we provide examples from the case companies we studied.

Reinforcement learning, particularly deep reinforcement learning, has made remarkable progress in recent years and is now used not only in simulators and games but is also making its way into embedded systems as another software-intensive domain. However, when implemented in a real-world context, reinforcement learning is typically shown to be fragile and incapable of adapting to dynamic environments. In this paper, we provide a novel dynamic reinforcement learning algorithm for adapting to complex industrial situations. We apply and validate our approach using a telecommunications use case. The proposed algorithm can dynamically adjust the position and antenna tilt of a drone-based base station to maintain reliable wireless connectivity for mission-critical users. When compared to traditional reinforcement learning approaches, the dynamic reinforcement learning algorithm improves the overall service performance of a drone-based base station by roughly 20%. Our results demonstrate that the algorithm can quickly evolve and continuously adapt to the complex dynamic industrial environment.

When developing software, it is vitally important to keep the level of technical debt down since, based on several studies, it has been well established that technical debt can lower the development productivity, decrease the developers' morale and compromise the overall quality of the software, among others. However, even if researchers and practitioners working in today's software development industry are quite familiar with the concept of technical debt and its related negative consequences, there has been no empirical research focusing specifically on how software managers actively communicate and manage the need to keep the level of technical debt as low as possible. This study aims to understand how software companies give incentives to manage technical debt. This is carried out by exploring how companies encourage and reward practitioners for actively keeping the level of technical debt down add whether the companies use any forcing or penalising initiatives when managing technical debt. As a first step, this paper reports the results of both an online survey providing quantitative data from 258 participants and interviews with 32 software practitioners. As a second step, this study sets out to specifically provide a detailed assessment of additional and in-depth analysis of technical debt management strategies based on an encouraging mindset and attitude from both managers and technical roles to understand how, when and by whom such strategies are adopted in practice. Our findings show that having a technical debt management strategy (specially based on encouragement) can significantly impact the amount of technical debt related to the software. The result indicates that there is considerable unfulfilled potential to influence how software practitioners can further limit and reduce technical debt by adopting a strategy based explicitly on an encouraging mindset from managers where they also specifically dedicate time and resources for technical debt remediation activities.

Deep learning (DL) based software systems are difficult to develop and maintain in industrial settings due to several challenges. Data management is one of the most prominent challenges which complicates DL in industrial deployments. DL models are data-hungry and require high-quality data. Therefore, the volume, variety, velocity, and quality of data cannot be compromised. This study aims to explore the data management challenges encountered by practitioners developing systems with DL components, identify the potential solutions from the literature and validate the solutions through a multiple case study. We identified 20 data management challenges experienced by DL practitioners through a multiple interpretive case study. Further, we identified 48 articles through a systematic literature review that discuss the solutions for the data management challenges. With the second round of multiple case study, we show that many of these solutions have limitations and are not used in practice due to a combination of four factors: high cost, lack of skill-set and infrastructure, inability to solve the problem completely, and incompatibility with certain DL use cases. Thus, data management for data-intensive DL models in production is complicated. Although the DL technology has achieved very promising results, there is still a significant need for further research in the field of data management to build high-quality datasets and streams that can be used for building production-ready DL systems. Furthermore, we have classified the data management challenges into four categories based on the availability of the solutions.

Data collected from in-service products play an important role in enabling software-intensive embedded systems suppliers to embrace data-driven practices. Data can be used in many different ways such as to continuously learn and improve the product, enhance post-deployment services, reduce operational cost or create a better user experience. While there is no shortage of possible use cases leveraging data from in-service products, software-intensive embedded systems companies struggle to continuously collect data from their in-service products. Often, data collection is done in an ad-hoc way and targeting specific use cases or needs. Besides, few studies have investigated data collection challenges in relation to the data dimensions, which are the minimum set of quantifiable data aspects that can define software-intensive embedded product data from a collection point of view. To help address data collection challenges, and to provide companies with guidance on how to improve this process, we conducted a case study at a large multinational telecommunications supplier focusing on data characteristics and collection challenges from the Radio Access Networks (RAN) products. We further investigated the relations of these challenges to the data dimensions to increase our understanding of how data dominions contribute to the challenges.

Randomised field experiments, such as A/B testing, have long been the gold standard for evaluating the value that new software brings to customers. However, running randomised field experiments is not always desired, possible or even ethical in the development of automotive embedded software. In the face of such restrictions, we propose the use of the Bayesian propensity score matching technique for causal inference of observational studies in the automotive domain. In this paper, we present a method based on the Bayesian propensity score matching framework, applied in the unique setting of automotive software engineering. This method is used to generate balanced control and treatment groups from an observational online evaluation and estimate causal treatment effects from the software changes, even with limited samples in the treatment group. We exemplify the method with a proof-of-concept in the automotive domain. In the example, we have a larger control (Nc = 1100) fleet of cars using the current software and a small treatment fleet (Nt = 38), in which we introduce a new software variant. We demonstrate a scenario that shipping of a new software to all users is restricted, as a result, a fully randomised experiment could not be conducted. Therefore, we utilised the Bayesian propensity score matching method with 14 observed covariates as inputs. The results show more balanced groups, suitable for estimating causal treatment effects from the collected observational data. We describe the method in detail and share our configuration. Furthermore, we discuss how can such a method be used for online evaluation of new software utilising small groups of samples.

The impact of the Artificial Intelligence revolution is undoubtedly substantial in our society, life, firms, and employment. With data being a critical element, organizations are working towards obtaining high-quality data to train their AI models. Although data, data management, and data pipelines are part of industrial practice even before the introduction of ML models, the significance of data increased further with the advent of ML models, which force data pipeline developers to go beyond the traditional focus on data quality. The objective of this study is to analyze the impact of ML use cases on data pipelines. We assume that the data pipelines that serve ML models are given more importance compared to the conventional data pipelines. We report on a study that we conducted by observing software teams at three companies as they develop both conventional(Non-ML) data pipelines and data pipelines that serve ML-based applications. We study six data pipelines from three companies and categorize them based on their criticality and purpose. Further, we identify the determinants that can be used to compare the development and maintenance of these data pipelines. Finally, we map these factors in a two-dimensional space to illustrate their importance on a scale of low, moderate, and high.

Continuous Deployment (CD) advocates for quick and frequent deployments of software to production. The goal is to bring new functionality as early as possible to users while learning from their usage. CD has emerged from web-based applications where it has been gaining traction over the past years. While CD is appealing for many software development organizations, empirical evidence on perceived benefits in software-intensive embedded systems is scarce. The objective of this paper is to identify perceived benefits after transitioning to continuous deployment from a long-cycle release and deployment process. To do that, a case study at a multinational telecommunication company was conducted focusing on large and complex embedded software; the Third Generation (3G) Radio Access Network (RAN) software.

The lack of labeled data is a major problem in both research and industrial settings since obtaining labels is often an expensive and time-consuming activity. In the past years, several machine learning algorithms were developed to assist and perform automated labeling in partially labeled datasets. While many of these algorithms are available in open-source packages, there is a lack of research that investigates how these algorithms compare to each other for different types of datasets and with different percentages of available labels. To address this problem, this paper empirically evaluates and compares seven algorithms for automated labeling in terms of their accuracy. We investigate how these algorithms perform in twelve different and well-known datasets with three different types of data, images, texts, and numerical values. We evaluate these algorithms under two different experimental conditions, with 10% and 50% labels of available labels in the dataset. Each algorithm, in each dataset for each experimental condition, is evaluated independently ten times with different random seeds. The results are analyzed and the algorithms are compared utilizing a Bayesian Bradley-Terry model. The results indicate that the active learning algorithms using the query strategies uncertainty sampling, QBC and random sampling are always the best algorithms. However, this comes with the expense of increased manual labeling effort. These results help machine learning practitioners in choosing optimal machine learning algorithms to label their data.

A/B experimentation is a known technique for data-driven product development and has demonstrated its value in web-facing businesses. With the digitalisation of the automotive industry, the focus in the industry is shifting towards software. For automotive embedded software to continuously improve, A/B experimentation is considered an important technique. However, the adoption of such a technique is not without challenge. In this paper, we present an architecture to enable A/B testing in automotive embedded software. The design addresses challenges that are unique to the automotive industry in a systematic fashion. Going from hypothesis to practice, our architecture was also applied in practice for running online experiments on a considerable scale. Furthermore, a case study approach was used to compare our proposal with state-of-practice in the automotive industry. We found our architecture design to be relevant and applicable in the efforts of adopting continuous A/B experiments in automotive embedded software.

With the development and the increasing interests in ML/DL fields, companies are eager to apply Machine Learning/Deep Learning approaches to increase service quality and customer experience. Federated Learning was implemented as an effective model training method for distributing and accelerating time-consuming model training while protecting user data privacy. However, common Federated Learning approaches, on the other hand, use a synchronous protocol to conduct model aggregation, which is inflexible and unable to adapt to rapidly changing environments and heterogeneous hardware settings in real-world scenarios. In this paper, we present an approach to real-time end-to-end Federated Learning combined with a novel asynchronous model aggregation protocol. Our method is validated in an industrial use case in the automotive domain, focusing on steering wheel angle prediction for autonomous driving. Our findings show that asynchronous Federated Learning can significantly improve the prediction performance of local edge models while maintaining the same level of accuracy as centralized machine learning. Furthermore, by using a sliding training window, the approach can minimize communication overhead, accelerate model training speed and consume real-time streaming data, proving high efficiency when deploying ML/DL components to heterogeneous real-world embedded systems.

Continuous Deployment is the practice to deploy software more frequently to customers and learn from their usage. The aim is to introduce new functionality and features in an additive way to customers as soon as possible. While Continuous Deployment is becoming popular among web and cloud-based software development organizations, the adoption of continuous deployment within the software-intensive embedded systems industry is still limited.In this paper, we conducted a case study at a multinational telecommunications company focusing on the Third Generation Radio Access Network (3G RAN) embedded software. The organization has transitioned to Continuous Deployment where the software’s deployment cycle has been reduced to 4 weeks from 24 weeks. The objective of this paper is to identify what does success means when transitioning to continuous deployment and the success factors that companies need to attend to when transitioning to continuous deployment in a large-scale embedded software.

In practice, supervised learning algorithms require fully labeled datasets to achieve the high accuracy demanded by current modern applications. However, in industrial settings supervised learning algorithms can perform poorly because of few labeled instances. Semi-supervised learning (SSL) is an automatic labeling approach that utilizes complete labels to infer missing labels in partially complete datasets. The high number of available SSL algorithms and the lack of systematic comparison between them leaves practitioners without guidelines to select the appropriate one for their application. Moreover, each SSL algorithm is often validated and evaluated in a small number of common datasets. However, there is no research that examines what datasets are suitable for comparing different SSL algorihtms. The purpose of this paper is to empirically evaluate the suitability of the datasets commonly used to evaluate and compare different SSL algorithms. We performed a simulation study using twelve datasets of three different datatypes (numerical, text, image) on thirteen different SSL algorithms. The contributions of this paper are two-fold. First, we propose the use of Bayesian congeneric item response theory model to assess the suitability of commonly used datasets. Second, we compare the different SSL algorithms using these datasets. The results show that with except of three datasets, the others have very low discrimination factors and are easily solved by the current algorithms. Additionally, the SSL algorithms have overlapping 90% credible intervals, indicating uncertainty in the difference between the accuracy of these SSL models. The paper concludes suggesting that researchers and practitioners should better consider the choice of datasets used for comparing SSL algorithms.

The adoption of continuous software engineering practices such as DevOps (Development and Operations) in business operations has contributed to significantly shorter software development and deployment cycles. Recently, the term MLOps (Machine Learning Operations) has gained increasing interest as a practice that brings together data scientists and operations teams. However, the adoption of MLOps in practice is still in its infancy and there are few common guidelines on how to effectively integrate it into existing software development practices. In this paper, we conduct a systematic literature review and a grey literature review to derive a framework that identifies the activities involved in the adoption of MLOps and the stages in which companies evolve as they become more mature and advanced. We validate this framework in three case companies and show how they have managed to adopt and integrate MLOps in their large-scale software development companies. The contribution of this paper is threefold. First, we review contemporary literature to provide an overview of the state-of-the-art in MLOps. Based on this review, we derive an MLOps framework that details the activities involved in the continuous development of machine learning models. Second, we present a maturity model in which we outline the different stages that companies go through in evolving their MLOps practices. Third, we validate our framework in three embedded systems case companies and map the companies to the stages in the maturity model.

A/B testing is gaining attention in the automotive sector as a promising tool to measure causal effects from software changes. Different from the web-facing businesses, where A/B testing has been well-established, the automotive domain often suffers from limited eligible users to participate in online experiments. To address this shortcoming, we present a method for designing balanced control and treatment groups so that sound conclusions can be drawn from experiments with considerably small sample sizes. While the Balance Match Weighted method has been used in other domains such as medicine, this is the first paper to apply and evaluate it in the context of software development. Furthermore, we describe the Balance Match Weighted method in detail and we conduct a case study together with an automotive manufacturer to apply the group design method in a fleet of vehicles. Finally, we present our case study in the automotive software engineering domain, as well as a discussion on the benefits and limitations of the A/B group design method.

Based on multiple series of interviews and workshops, this paper presents the MaLET model – a representation of the typical evolution path for companies who successfully adopt exploratory testing. The model provides a step-by-step approach to systematically improve exploratory testing over time, and shows how and why capabilities may regress. The MaLET model was validated through a series of interviews with 20 interviewees from eight case study companies in separate industry segments. The interviews also revealed examples on improvement initiatives that had failed in the companies, showing that improvement initiatives tend to fail if they are not planned in an order corresponding to the maturity levels in the model. The MaLET model was well received by the interviewees during the validation, describing the model as sound, relevant and useful in practice.

In recent years, with more edge devices being put into use, the amount of data that is created, transmitted and stored is increasing exponentially. Moreover, due to the development of machine learning algorithms, modern software-intensive systems are able to take advantage of the data to further improve their service quality. However, it is expensive and inefficient to transmit large amounts of data to a central location for the purpose of training and deploying machine learning models. Data transfer from edge devices across the globe to central locations may also raise privacy and concerns related to local data regulations. As a distributed learning approach, Federated Learning has been introduced to tackle those challenges. Since Federated Learning simply exchanges locally trained machine learning models rather than the entire data set throughout the training process, the method not only protects user data privacy but also improves model training efficiency. In this paper, we have investigated an advanced machine learning algorithm, Deep Neural Decision Forests (DNDF), which unites classification trees with the representation learning functionality from deep convolutional neural networks. In this paper, we propose a novel algorithm, AF-DNDF which extends DNDF with an asynchronous federated aggregation protocol. Based on the local quality of each classification tree, our architecture can select and combine the optimal groups of decision trees from multiple local devices. The introduction of the asynchronous protocol enables the algorithm to be deployed in the industrial context with heterogeneous hardware settings. Our AF-DNDF architecture is validated in an automotive industrial use case focusing on road objects recognition and demonstrated by an empirical experiment with two different data sets. The experimental results show that our AF-DNDF algorithm significantly reduces the communication overhead and accelerates model training speed without sacrificing model classification performance. The algorithm can reach the same classification accuracy as the commonly used centralized machine learning methods but also greatly improve local edge model quality.

In recent years, with the development of computation capability in devices, companies are eager to investigate and utilize suitable ML/DL methods to improve their service quality. However, with the traditional learning strategy, companies need to first build up a powerful data center to collect and analyze data from the edge and then perform centralized model training, which turns out to be inefficient. Federated Learning has been introduced to solve this challenge. Because of its characteristics such as model-only exchange and parallel training, the technique can not only preserve user data privacy but also accelerate model training speed. The method can easily handle real-time data generated from the edge without taking up a lot of valuable network transmission resources. In this paper, we introduce an approach to end-to-end on-device Machine Learning by utilizing Federated Learning. We validate our approach with an important industrial use case in the field of autonomous driving vehicles, the wheel steering angle prediction. Our results show that Federated Learning can significantly improve the quality of local edge models and also reach the same accuracy level as compared to the traditional centralized Machine Learning approach without its negative effects. Furthermore, Federated Learning can accelerate model training speed and reduce the communication overhead, which proves that this approach has great strength when deploying ML/DL components to various real-world embedded systems.

Abstract With digitalization and with technologies such as software, data, and artificial intelligence, companies in the embedded systems domain are experiencing a rapid transformation of their conventional businesses. While the physical products and associated product sales provide the core revenue, these are increasingly being complemented with service offerings, new data‐driven services, and digital products that allow for continuous value creation and delivery to customers. However, although there is significant research on digitalization and digital transformation, few studies highlight the specific needs of embedded systems companies and what it takes to transform from a traditional towards a digital company within business domains characterized by high complexity, hardware dependencies, and safety‐critical system functionality. In this paper, we capture the difference between what constitutes a traditional and a digital company and we detail the typical evolution path embedded systems companies take when transitioning towards becoming digital companies.

Frequentist statistical methods, such as hypothesis testing, are standard practice in papers that provide benchmark comparisons. Unfortunately, these methods have often been misused, e.g., without testing for their statistical test assumptions or without controlling for family-wise errors in multiple group comparisons, among several other problems. Bayesian Data Analysis (BDA) addresses many of the previously mentioned shortcomings but its use is not widely spread in the analysis of empirical data in the evolutionary computing community. This paper provides three main contributions. First, we motivate the need for utilizing Bayesian data analysis and provide an overview of this topic. Second, we discuss the practical aspects of BDA to ensure that our models are valid and the results transparent. Finally, we provide five statistical models that can be used to answer multiple research questions. The online appendix provides a step-by-step guide on how to perform the analysis of the models discussed in this paper, including the code for the statistical models, the data transformations and the discussed tables and figures.

The demands on software continues to increase through the constant addition of functionalities and high expectations from users. In particular, performance has been the focus in many projects with the goal of fulfilling complex and hard requirements across a variety of domains. One way to achieve satisfactory levels of performance is through heterogeneous computing, i.e., systems that contain more than one type of processing unit, such as CPUs, GPUs, and FPGAs. However, applications are typically designed to be executed on CPUs, and re-architecting software for execution on such heterogeneous hardware architectures entails several challenges that must be addressed. In this paper, we propose a framework that supports engineers in the process of making architectural decisions to re-architect software for execution on heterogeneous platforms. We present several relevant aspects that should be addressed in the process, along with suggestions on how to create design solutions using different existing approaches. The framework was developed based on multiple interactions with three industrial partners and evaluated through a computer vision application in the automotive domain.

In order to distinguish themselves from their competitors, software service providers constantly try to assess and improve customer satisfaction. However, measuring customer satisfaction in a continuous way is often time and cost intensive, or requires effort on the customer side. Especially in B2B contexts, a continuous assessment of customer satisfaction is difficult to achieve due to potential restrictions and complex provider-customer-end user setups. While concepts such as web usage mining enable software providers to get a deep understanding of how their products are used, its application to quantitatively measure customer satisfaction has not yet been studied in greater detail. For that reason, our study aims at combining existing knowledge on customer satisfaction, web usage mining, and B2B service characteristics to derive a model that enables an automated calculation of quantitative customer satisfaction scores. We apply web usage mining to validate these scores and to compare the usage behavior of satisfied and dissatisfied customers. This approach is based on domain-specific service quality and web usage metrics and is, therefore, suitable for continuous measurements without requiring active customer participation. The applicability of the model is validated by instantiating it in a real-world B2B online platform.

Machine learning and deep learning techniques are becoming increasingly popular and critical for companies as part of their systems. However, although the development and prototyping of ML/DL systems are common across companies, the transition from prototype to production-quality deployment models are challenging. One of the key challenges is how to determine the selection of an optimal architecture for AI deployment. Based on our previous research, and to offer support and guidance to practitioners, we developed a framework in which we present five architectural alternatives for AI deployment ranging from centralized to fully decentralized edge architectures. As part of our research, we validated the framework in software-intensive embedded system companies and identified key challenges they face when deploying ML/DL models. In this paper, and to further advance our research on this topic, we identify factors that help practitioners determine what architecture to select for the ML/D L model deployment. For this, we conducted a follow-up study involving interviews and workshops in seven case companies in the embedded systems domain. Based on our findings, we identify three key factors and develop a framework in which we outline how prioritization and trade-offs between these results in certain architecture. The contribution of the paper is threefold. First, we identify key factors critical for AI system deployment. Second, we present the architecture selection framework that explains how prioritization and trade-offs between key factors result in the selection of a certain architecture. Third, we discuss additional factors that mayor may not influence the selection of an optimal architecture.

Process Debt, like Technical Debt, can be the source of short-term benefits but often is harmful in the long term for a software organization. Nonetheless, information about Process Debt is scarce in current literature. We conducted an exploratory study of Process Debt in four international organizations by interviewing 16 practitioners. The findings show that Process Debt can be a harmful phenomenon that needs attention and new practices, as it is different from Technical Debt. We provide an initial framework, composed by a definition and a conceptual model for Process Debt, showing types, causes, consequences, and debt accumulation patterns.

Data pipelines play an important role throughout the data management process. It automates the steps ranging from data generation to data reception thereby reducing the human intervention. A failure or fault in a single step of a data pipeline has cascading effects that might result in hours of manual intervention and clean-up. Data pipeline failure due to faults at different stages of data pipelines is a common challenge that eventually leads to significant performance degradation of data-intensive systems. To ensure early detection of these faults and to increase the quality of the data products, continuous monitoring and fault detection mechanism should be included in the data pipeline. In this study, we have explored the need for incorporating automated fault detection mechanisms and mitigation strategies at different stages of the data pipeline. Further, we identified faults at different stages of the data pipeline and possible mitigation strategies that can be adopted for reducing the impact of data pipeline faults thereby improving the quality of data products. The idea of incorporating fault detection and mitigation strategies is validated by realizing a small part of the data pipeline using action research in the analytics team at a large software-intensive organization within the telecommunication domain.

Machine Learning (ML) and Artificial Intelligence (AI) have increasingly gained attention in research and industry. Federated Learning, as an approach to distributed learning, shows its potential with the increasing number of devices on the edge and the development of computing power. However, most of the current Federated Learning systems apply a single-server centralized architecture, which may cause several critical problems, such as the single-point of failure as well as scaling and performance problems. In this paper, we propose and compare four architecture alternatives for a Federated Learning system, i.e. centralized, hierarchical, regional and decentralized architectures. We conduct the study by using two well-known data sets and measuring several system performance metrics for all four alternatives. Our results suggest scenarios and use cases which are suitable for each alternative. In addition, we investigate the trade-off between communication latency, model evolution time and the model classification performance, which is crucial to applying the results into real-world industrial systems.

The most important way to achieve higher performance in computer systems is through heterogeneous computing, i.e., by adopting hardware platforms containing more than one type of processor, such as CPUs, GPUs, and FPGAs. Several types of algorithms can be executed significantly faster on a heterogeneous platform. However, migrating CPU-executable software to other types of execution platforms poses a number of challenges to software engineering. Significant efforts are required in such type of migration, particularly for re-architecting and re-implementing the software. Further, optimizing it in terms of performance and other runtime properties can be very challenging, making the process complex, expensive, and error-prone. Therefore, a systematic approach based on explicit and justified architectural decisions is needed for a successful refactoring process from a homogeneous to a heterogeneous platform. In this paper, we propose a decision framework that supports engineers when refactoring software systems to accommodate heterogeneous platforms. It includes the assessment of important factors in order to minimize the risk of recurrent problems in the process. Through a set of questions, practitioners are able to formulate answers that will help in making appropriate architectural decisions to accommodate heterogeneous platforms. The contents of the framework have been developed and evolved based on discussions with architects and developers in the automotive domain.

System logs perform a critical function in software-intensive systems as logs record the state of the system and significant events in the system at important points in time. Unfortunately, log entries are typically created in an ad-hoc, unstructured and uncoordinated fashion, limiting their usefulness for analytics and machine learning. In a DevOps environment, especially, unmanaged evolution in log data structure causes frequent disruption of operations in automated data pipelines, dashboards and analytics. In this paper, we present the main challenges of contemporary approaches to generating, storing and managing the evolution of system logs data for large, complex, software-intensive systems based on an in-depth case study at a world-leading telecommunications company. Second, we present an approach for generating and managing the evolution of log data in a DevOps environment that does not suffer from the aforementioned challenges and is optimized for use in machine learning. Third, we provide validation of the approach based on expert interviews that confirm that the approach addresses the identified challenges and problems.

In recent years, the application of artificial intelligence (AI) has become an integral part of a wide range of areas, including software engineering. By analyzing various data sources generated in software engineering, it can provide valuable insights into customer behavior, product performance, bugs and errors, and many more. In practice, however, AI for software analytics and business intelligence often gets stuck in a prototypical stage and the results are rarely used to make decisions based on data. To understand the underlying root causes of this phenomenon, we conduct both an explanatory case study and a survey on the challenges of realizing and utilizing artificial intelligence in the context of software-intensive businesses. As a result, we identify a vicious circle that prevents practitioners from moving from prototypical analytics to continuous and productively usable software analytics and business intelligence based on AI.

Since the advent of mobile computing and IoT, a large amount of data is distributed around the world. Companies are increasingly experimenting with innovative ways of implementing edge/cloud (re)training of AI systems to exploit large quantities of data to optimize their business value. Despite the obvious benefits, companies face challenges as the decision on how to implement edge/cloud (re)training depends on factors such as the task intent, the amount of data needed for (re)training, edge-to-cloud data transfer, the available computing and memory resources. Based on action research in a software-intensive embedded systems company where we study multiple use cases as well as insights from our previous collaborations with industry, we develop a generic framework consisting of five architectural alternatives to deploy AI on the edge utilizing transfer learning. We validate the framework in four additional case companies and present the challenges they face in selecting the optimal architecture. The contribution of the paper is threefold. First, we develop a generic framework consisting of five architectural alternatives ranging from a centralized architecture where cloud (re)training is given priority to a decentralized architecture where edge (re)training is instead given priority. Second, we validate the framework in a qualitative interview study with four additional case companies. As an outcome of validation study, we present two variants to the architectural alternatives identified as part of the framework. Finally, we identify the key challenges that experts face in selecting an ideal architectural alternative.

Over the past 15 years, A/B testing has been a critical tool for accurate prioritization of development efforts in online and web-facing companies. As automotive companies progress on their digitalization process, A/B testing and other experimentation techniques start to be adopted. However, specific characteristics of the automotive software industry create additional challenges to the successful adoption of A/B testing. Recently, research has been conducted to investigate the challenges and opportunities for experimentation techniques in the automotive and more generally in the embedded systems domain. However, despite the collaboration with industry, previous research was based on either hypothesized or toy scenarios in companies seeking, but not yet running experimentation. Utilizing a case study method, we investigate the challenges of adopting A/B testing in two large-scale automotive companies that are currently running or preparing for their first A/B testing. The contribution of this paper is two-fold. First, we present our main findings in terms of the challenges of real A/B testing iterations in automotive vehicles. Second, we present the current, potential solutions and lessons learned from applying A/B testing in the automotive domain.

Data driven and experimental development practices provide effective means for companies to adopt a customer and market-centric way-of-working. In online companies, controlled experimentation is the primary technique to measure how customers respond to variants of deployed software. Over the recent years, and due to increasing connectivity and data collection from products in the field, these practices are being adopted also in software-intensive embedded systems companies. In these companies, experiments are run on selected instances of the system or as comparisons of previously computed data to ensure value delivery to customers, improve quality and explore new value propositions. However, to utilize the benefits of data- driven and experimental development practices, companies need to define what value factors to optimize for. For highly complex embedded systems with thousands of parameters, and with people at different levels in the organization having different opinions about the value of features, this is a challenging task. In this paper, we report on longitudinal multi-case study research in which we explore value modeling as a technique to help people in development, in product management and on the business level to align interests and agree on value factors. Based on this work, we identify five purposes of value modeling and how this technique helps accelerate critical activities in an organization. The contribution of this paper is three-fold. First, we provide empirical evidence for how value modeling is an effective technique to help companies define what to optimize for. Second, we identify five purposes of value modeling. Third, we identify the key challenges that the case companies experience when applying value modeling.

The following topics are dealt with: software development management; software quality; software engineering; software maintenance; project management; formal specification; public domain software; learning (artificial intelligence); software prototyping; software architecture.

When developing software, it is vitally important to keep the level of technical debt down since it is well established from several studies that technical debt can, e.g., lower the development productivity, decrease the developers' morale, and compromise the overall quality of the software. However, even if researchers and practitioners working in today's software development industry are quite familiar with the concept of technical debt and its related negative consequences, there has been no empirical research focusing specifically on how software managers actively communicate and manage the need to keep the level of technical debt as low as possible.

Automatic labeling is a type of classification problem. Classification has been studied with the help of statistical methods for a long time. With the explosion of new better computer processing units (CPUs) and graphical processing units (GPUs) the interest in machine learning has grown exponentially and we can use both statistical learning algorithms as well as deep neural networks (DNNs) to solve the classification tasks. Classification is a supervised machine learning problem and there exists a large amount of methodology for performing such task. However, it is very rare in industrial applications that data is fully labeled which is why we need good methodology to obtain error-free labels. The purpose of this paper is to examine the current literature on how to perform labeling using ML, we will compare these models in terms of popularity and on what datatypes they are used on. We performed a systematic literature review of empirical studies for machine learning for labeling. We identified 43 primary studies relevant to our search. From this we were able to determine the most common machine learning models for labeling. Lack of unlabeled instances is a major problem for industry as supervised learning is the most widely used. Obtaining labels is costly in terms of labor and financial costs. Based on our findings in this review we present alternate ways for labeling data for use in supervised learning tasks.

Artificial Intelligence is becoming increasingly popular with organizations due to the success of Machine Learning and Deep Learning techniques. Using these techniques, data scientists learn from vast amounts of data to enhance behaviour in software-intensive systems. Despite the attractiveness of these techniques, however, there is a lack of systematic and structured design process for developing ML/DL models. The study uses a multiple-case study approach to explore the different activities and challenges data scientists face when developing ML/DL models in software-intensive embedded systems. In addition, we have identified seven different phases in the proposed design process leading to effective model development based on the case study. Iterations identified between phases and events which trigger these iterations optimize the design process for ML/DL models. Lessons learned from this study allow data scientists and engineers to develop high-performance ML/DL models and also bridge the gap between high demand and low supply of data scientists.

Continuous experimentation (CE) refers to a group of practices used by software companies to rapidly assess the usage, value and performance of deployed software using data collected from customers and the deployed system. Despite its increasing popularity in the development of web-facing applications, CE has not been discussed in the development process of business-to-business (B2B) mission-critical systems.

The collection of high-quality data provides a key competitive advantage to companies in their decision-making process. It helps to understand customer behavior and enables the usage and deployment of new technologies based on machine learning. However, the process from collecting the data, to clean and process it to be used by data scientists and applications is often manual, non-optimized and error-prone. This increases the time that the data takes to deliver value for the business. To reduce this time companies are looking into automation and validation of the data processes. Data processes are the operational side of data analytic workflow.

Abstract Digitalization is transforming industry to an extent that we have only seen the beginnings of. Across domains, companies experience rapid changes to their existing practices due to new technologies and new entrants that current businesses. While digitalization brings endless opportunities, it comes with challenges that require companies to strategically engage with partners in their surrounding ecosystems. In this paper, we study how companies in the embedded systems domain experience the process of transitioning from product‐based companies to businesses where software, data, and artificial intelligence (AI) play an increasingly important role. To manage this, these companies need to evolve their existing ecosystems while at the same time create new ecosystems around new technologies. This involves maintaining existing technologies such as mechanics and electronics while at the same time expanding these with software, data, and AI. We provide a strategic decision framework that helps software‐intensive embedded systems companies to successfully navigate the digital transformation. We do this in two steps. First, we present three models that provide the technical content of the strategic decision framework. Second, we provide an overview of the strategic alternatives that incumbents and new entrants have available when existing technologies are commoditizing and new technologies are introduced.

One way to improve the performance of embedded systems is through heterogeneous platforms, i.e., using hardware containing more than one type of processor, like CPU + GPU or CPU + FPGA. This approach has shown improved performance, particularly in the domain of artificial intelligence, in which computationally demanding models must be trained and executed. However, these computational environments pose various challenges to software engineering, since applications must be designed and developed differently while accounting for target hardware architectures that are inherently different. Companies interested in migrating to heterogeneous platforms must be aware of the changes to the software development processes that are required to accommodate such solution. In this paper, we conducted a multiple case study that aims to discover the challenges and common practices when developing software for heterogeneous platforms in industrial contexts. First, we organized semi-structured interviews with companies on the automotive, automation, and telecommunication domains. Then, we analyzed and structured the data in order to create a classification describing the software engineering challenges faced by companies using heterogeneous platforms. The companies involved in this study are in different stages of maturity concerning the use of heterogeneous platforms. Thus, the classification takes into consideration these levels to describe the challenges accordingly.

In recent years in the software industry, the use of safety-critical software is increasing at a rapid rate. However, little is known about the relationship between safety-critical regulations and the management of technical debt. The research is based on interviews with 19 practitioners working in different safety-critical domains implementing software according to different safety regulation standards. The results are three-fold. First, the result shows that performing technical debt refactoring tasks in safety-critical software requires several additional activities and costs, compared to non-safety-critical software. This study has also identified several negative effects due to the impact of these regulatory requirements. Second, the results show that the safety-critical regulations strengthen the implementation of both source code and architecture and thereby initially limit the introduction of technical debt. However, at the same time, the regulations also force the software companies to perform later suboptimal work-around solutions that are counterproductive in achieving a high-quality software since the regulations constrain the possibility of performing optimal TD refactoring activities. Third, the result shows that technical debt refactoring decisions are heavily weighed on the costs associated with the application's recertification process and that these decisions seldom include the benefits of the refactoring activities in a structured way.

A wide variety of tools to monitor and track software systems, such as websites or smartphone applications, during runtime already exists. However, their aggregated results are often not sufficient to answer questions on a product management level since these questions address several levels of complexity and abstractions, and tend to be formulated on a rather high level, for instance concerning the efficiency of their website structure for their users. A straightforward mapping between low level metrics and high level insights is typically not possible. This causes a gap that makes it challenging to continuously provide quantitative high-level insights in real-time. In order to address this challenge, we conducted a study within three distinct platforms and products, and propose a model based on our results. After defining a case for each of the independent platforms and products, we implemented a process to measure high level insights using low level metrics for each of these cases. Next, we compared the procedures and steps that were taken in each of the cases and derived a model that describes a generic approach how to utilize and process data in order to gain higher level insights. Our model structures the steps from data to knowledge over different levels of complexity and abstraction, namely operational, tactical, and strategic. Thereby, the knowledge acquired in each phase serves as input in the next phase which increases the measurable level of complexity with each iteration. Since the steps in our model are specifically arranged as a pipeline, it enables practitioners to automate a continuous and quantitative measurement of high level insights in real-time.

Deep learning is one of the most exciting and fast-growing techniques in Artificial Intelligence. The unique capacity of deep learning models to automatically learn patterns from the data differentiates it from other machine learning techniques. Deep learning is responsible for a significant number of recent breakthroughs in AI. However, deep learning models are highly dependent on the underlying data. So, consistency, accuracy, and completeness of data is essential for a deep learning model. Thus, data management principles and practices need to be adopted throughout the development process of deep learning models. The objective of this study is to identify and categorise data management challenges faced by practitioners in different stages of end-to-end development. In this paper, a case study approach is employed to explore the data management issues faced by practitioners across various domains when they use real-world data for training and deploying deep learning models. Our case study is intended to provide valuable insights to the deep learning community as well as for data scientists to guide discussion and future research in applied deep learning with real-world data.

This paper identifies three improvement areas related to system design and architecture, where an organization can change to better support continuous integration and continuous delivery: “The product's architecture”, “Ways to work with system design and architecture”, and “The role of the architect”. The three improvement areas are based on a literature review, two series of interviews and a cross-company workshop with three case-study companies. Furthermore, the paper proposes three actionable strategies corresponding to the three identified improvement areas: “Systems with a modular and loosely coupled architecture”, “A balanced approach where system design and architecture is focused on the system's most important characteristics”, and “Architects shifting perspective from control to facilitation”.

Remediation of technical debt through regular refactoring initiatives is considered vital for the software system's long and healthy life. However, since today's software companies face increasing pressure to deliver customer value continuously, the balance between spending developer time, effort, and resources on implementing new features or spending it on refactoring of technical debt becomes vital. The goal of this study is to explore how the prioritization of technical debt is carried out by practitioners within today's software industry. This study also investigates what factors influence the prioritization process and its related challenges. This paper reports the results of surveying 17 software practitioners, together with follow-up interviews with them. Our results show that there is no uniform way of prioritizing technical debt and that it is commonly done reactively without applying any explicit strategies. Often, technical debt issues are managed and prioritized in a shadow backlog, separate from the official sprint backlog. This study was also able to identify several different challenges related to prioritizing technical debt, such as the lack of quantitative information about the technical debt items and that the refactoring of technical debt issues competes with the implementation of customer requirements.

The digital transformation requires companies to change their operating system from a traditional to a digital one. This implies changes to business models, the business ecosystem, the architecture of products, the ways of working, the structure of the organization as well as the basic organizational culture. In this paper, we present our perspective on the operating system that digital companies employ consisting of five dimensions, i.e. speed, data, artificial intelligence, empowerment and ecosystems. For each of these dimension, we present the typical stages that companies evolve through while undergoing the transformation. The contribution of the paper is that it provides a holistic, integrated framework that allows companies to define desired state, assess current state and plan the necessary steps to transition.

Abstract The shift from sequential to agile software development originates from relatively small and co-located teams but soon gained prominence in larger organizations. How to apply and scale agile practices to fit the needs of larger projects has been studied to quite an extent in previous research. However, scaling agile beyond organizational boundaries, for instance in a software ecosystem context, raises additional challenges that existing studies and approaches do not yet investigate or address in great detail. For that reason, we conducted a case study in two software ecosystems that comprise several agile actors from different organizations and, thereby, scale development across organizational boundaries, in order to elaborate and understand their coordination challenges. Our results indicate that most of the identified challenges are caused by long communication paths and a lack of established processes to facilitate these paths. As a result, the participants in our study, among others, experience insufficient responsivity, insufficient communication of prioritizations and deliverables, and alterations or loss of information. As a consequence, agile practices need to be extended to fit the identified needs.

Abstract Artificial intelligence enabled systems have been an inevitable part of everyday life. However, efficient software engineering principles and processes need to be considered and extended when developing AI- enabled systems. The objective of this study is to identify and classify software engineering challenges that are faced by different companies when developing software-intensive systems that incorporate machine learning components. Using case study approach, we explored the development of machine learning systems from six different companies across various domains and identified main software engineering challenges. The challenges are mapped into a proposed taxonomy that depicts the evolution of use of ML components in software-intensive system in industrial settings. Our study provides insights to software engineering community and research to guide discussions and future research into applied machine learning.

Abstract Based on 25 interviews with participants from four case study companies that develop large‐scale software embedded systems, this paper presents the Test Activity Stakeholders (TAS) model. The TAS model shows how the continuous integration and delivery pipeline can be designed to include test activities that support four stakeholder interests: “Check changes,” “Secure stability,” “Measure progress,” and “Verify compliance.” The model is developed to show how each of the stakeholder interests are best supported by unit/component tests or system tests, by automated testing or manual testing and by tests executed in simulated environments or on real hardware. The TAS model may serve as a starting point for companies when evaluating and designing their continuous integration and delivery pipeline. The validation of the TAS model included twelve individuals from three case study companies. The validation showed that the TAS model is actionable and useful in practice, enabling the identification of stakeholders and showing where improvement efforts should be focused in order to support all stakeholder interests in the continuous integration and delivery pipeline.

The ever-increasing complexity of software and rapidly changing development environments continues to drive the evolution of new technologies, techniques, and tools. This special issue, Software Engineering in Practice, provides the software engineering community with a valuable collection of current, high-quality research articles that explore topics driven by real problems in the industry. The inspiration for this special issue has drawn upon the ICSE Software Engineering in Practice (ICSE SEIP 2018) Track, 1 part of the Industry Program at the 40th International Conference on Software Engineering. 2 The ICSE SEIP Track provides a premier venue for researchers and practitioners to discuss innovations and solutions to concrete software engineering problems. The Guest Co-editor team for this special issue is an international collaboration involving the co-organizers of the ICSE SEIP 2018 Track, Frances Paulisch and Jan Bosch, and an editor of the Journal of Software: Practice and Experience 3 (JSPE), Kendra M. L. Cooper. Methodology The Call for Papers was designed to encourage submissions that presented novel and innovative ideas that broadly spanned the software engineering discipline—ideas that provided rigorously validated solutions for real problems encountered by practitioners. In order to promote an inclusive environment, the call was broadly disseminated as an open call; it was advertised on the JSPE and the ICSE SEIP 2018 websites, established software engineering newsgroups (eg, SE World), and conference announcement sites (eg, WikiCFP), in addition to numerous professional and social media platforms. The call required submissions be original manuscripts that had not been previously published and were also not under consideration for publication elsewhere. Submissions of research article, survey papers, short communication, and extended conference papers were welcome; extended conference papers were required to include at least 30% additional novel contributions. The response from the software engineering community was enthusiastic: the special issue received 41 submissions. Submissions featured international collaborations from researchers in academia and industry, case studies from industry, and the use of open source data sets and systems provided by the broader community. The submissions were reviewed according to the JSPE standards, with a goal of publishing the online version of the articles in a timely fashion. Ultimately, nine articles were selected for the special issue. Overviews of these accepted manuscripts are presented below, organized into three groups. Overview of the collection Novel contributions in the area of data analytics are explored in some of the papers in the special issue. Astekin et al present a framework for automated analysis of large system logs to detect anomalies. Karna et al employ data mining methods on the software engineering process itself and use these techniques for effort estimation. Marijan et al introduce a novel learning algorithm to optimize the continuous integration and testing practice. In addition, Martens et al investigate the problem of data migration between legacy and newly developed IT systems. Research problems centered on mobile application code are the focus of two papers in the special issue. Ochoa et al apply static analysis techniques on Android applications to detect context leaks in the code. Wang et al also study mobile applications; these authors report on approaches to obfuscate mobile application code with the intent of impeding efforts to reverse engineering applications. Techniques based on the model-driven development paradigm are presented in several of the papers in the special issue. Nooraei and Ajoudanian present a framework to ensure the on-going integrity of the transformation between the platform-independent model (PIM) and the corresponding platform-specific model (PSM) of a system via regression testing. Markthaler et al present a model-driven method for the automotive domain that spans multiple layers of software engineering artifacts; on-going consistency checking between the layers is supported in addition to the generation and re-generation of test cases. O'Shea et al present a three-layer design model for a virtualized test automation framework; a key feature of this model is the virtualized infrastructure layer. We would like to extend our warmest thanks to all the authors who submitted their manuscripts, the anonymous reviewers who provided timely and high-quality review comments in their generous service to the community, and the editorial board and personnel of the Journal of Software: Practice and Experience.

Surprisingly promising results have been achieved by deep learning (DL) systems in recent years. Many of these achievements have been reached in academic settings, or by large technology companies with highly skilled research groups and advanced supporting infrastructure. For companies without large research groups or advanced infrastructure, building high-quality production-ready systems with DL components has proven challenging. There is a clear lack of well-functioning tools and best practices for building DL systems. It is the goal of this research to identify what the main challenges are, by applying an interpretive research approach in close collaboration with companies of varying size and type. A set of seven projects have been selected to describe the potential with this new technology and to identify associated main challenges. A set of 12 main challenges has been identified and categorized into the three areas of development, production, and organizational challenges. Furthermore, a mapping between the challenges and the projects is defined, together with selected motivating descriptions of how and why the challenges apply to specific projects. Compared to other areas such as software engineering or database technologies, it is clear that DL is still rather immature and in need of further work to facilitate development of high-quality systems. The challenges identified in this paper can be used to guide future research by the software engineering and DL communities. Together, we could enable a large number of companies to start taking advantage of the high potential of the DL technology.

Online Controlled Experiments (OCEs) are the norm in data-driven software companies because of the benefits they provide for building and deploying software. Product teams experiment to accurately learn whether the changes that they do to their products (e.g. adding new features) cause any impact (e.g. customers use them more frequently). Experiments also help reduce the risk from deploying software by minimizing the magnitude and duration of harm caused by software bugs, allowing software to be shipped more frequently. To make informed decisions in product development, experiment analysis needs to be granular with a large number of metrics over heterogeneous devices and audiences. Discovering experiment insights by hand, however, can be cumbersome. In this paper, and based on case study research at a large-scale software development company with a long tradition of experimentation, we (1) describe the standard process of experiment analysis, and (2) introduce an artifact to improve the effectiveness and comprehensiveness of this process.

Online Controlled Experiments (OCEs, aka A/B tests) are one of the most powerful methods for measuring how much value new features and changes deployed to software products bring to users. Companies like Microsoft, Amazon, and Booking.com report the ability to conduct thousands of OCEs every year. However, the competences of the remainder of the online software industry remain unknown. The main objective of this paper is to reveal the current state of A/B testing maturity in the software industry based on a maturity model from our previous research. We base our findings on 44 responses from an online empirical survey. Our main contribution of this paper is the current state of experimentation maturity as operationalized by the ExG model for a convenience sample of companies doing online controlled experiments. Our findings show that, among others, companies typically develop in-house experimentation platforms, that these platforms are of various levels of maturity, and that designing key metrics - Overall Evaluation Criteria - remains the key challenge for successful experimentation.

Empowerment is based on the belief that employees have the ability, and the desire, to shoulder more responsibility and perform better when given freedom. In an empowered organization, authority is given to employees with the intent to increase responsiveness to customers, improve decision-making power and to increase team motivation and skills. However, while most studies picture empowerment as the "ideal state" and the place where all organizations strive to be, our research shows that fully empowered teams without strategic guidance suffer from a number of problems. Based on multi-case study research in eleven software-intensive companies, we see that companies need to allow for different levels of empowerment depending on what they aim to achieve, characteristics of the industry domain, the business model and other factors, and that strategic guidance is critical to set direction and for avoiding chaos. To help companies approach the optimal level of empowerment, we provide a framework consisting of two inter-connected models that help companies to, rather than staying in their current hierarchical structures, transition to a level of empowerment that maximizes business value and performance.

The role of software in society and in industry in particular continues to grow exponentially.Most companies either have or are in the process of adoption continuous deployment of their software at products in the field and collect data concerning the performance of their systems.The continuous, fast feedback loops that companies now have available allow for a fundamentally different way of organizing.In fact, based on our work with dozens of companies, we have come to the conclusion that companies are moving towards a new, digital operating system.In this paper, we first present the key elements of the digital operating system and then discuss some of the challenges companies experience during the transformation.

Software companies need to continuously deliver customer value, both from a short- and long-term perspective. However, software development can be impeded by what has been described as Technical Debt (TD). The aim of this study is to explore the negative consequences of TD in terms of wasted software development time. This study also investigates on which additional activities this wasted time is spent and whether different types of TD impact the wasted time differently. This study also sets out to examine the benefits of tracking and communicating the amount of wasted time, both from a developer's and manager's perspective. This paper reports the results of a longitudinal study, surveying 43 software developers, together with follow-up interviews with 16 industrial software practitioners. The analysis of the reported wasted time revealed that developers waste, on average, 23% of their development time due to TD and that they are frequently forced to introduce new TD due to already existing TD. The most common activity on which additional time is spent is performing additional testing.

Software startups are typically under extreme pressure to get to market quickly with limited resources and high uncertainty. This pressure and uncertainty is likely to cause startups to accumulate technical debt as they make decisions that are more focused on the short-term than the long-term health of the codebase. However, most research on technical debt has been focused on more mature software teams, who may have less pressure and, therefore, reason about technical debt very differently than software startups. In this study, we seek to understand the organizational factors that lead to and the benefits and challenges associated with the intentional accumulation of technical debt in software startups. We interviewed 16 professionals involved in seven different software startups. We find that the startup phase, the experience of the developers, software knowledge of the founders, and level of employee growth are some of the organizational factors that influence the intentional accumulation of technical debt. In addition, we find the software startups are typically driven to achieve a "good enough level," and this guides the amount of technical debt that they intentionally accumulate to balance the benefits of speed to market and reduced resources with the challenges of later addressing technical debt.

This extended abstract summarizes an article, which has been published in Information and Software Technology and was selected for the Journal-First presentations at the International Conference on Software and System Process (ICSSP 2018).

Online controlled experiments are extensively used by web-facing companies to validate and optimize their systems, providing a competitive advantage in their business. As the number of experiments scale, companies aim to invest their experimentation resources in larger feature changes and leave the automated techniques to optimize smaller features. Optimization experiments in the continuous space are encompassed in the many-armed bandits class of problems. Although previous research provides algorithms for solving this class of problems, these algorithms were not implemented in real-world online experimentation problems and do not consider the application constraints, such as time to compute a solution, selection of a best arm and the estimation of the mean-reward function. This work discusses the online experiments in context of the many-armed bandits class of problems and provides three main contributions: (1) an algorithm modification to include online experiments constraints, (2) implementation of this algorithm in an industrial setting in collaboration with Sony Mobile, and (3) statistical evidence that supports the modification of the algorithm for online experiments scenarios. These contributions support the relevance of the LG-HOO algorithm in the context of optimization experiments and show how the algorithm can be used to support continuous optimization of online systems in stochastic scenarios.

We are living in the most exciting time in the history of mankind. The last century has seen unprecedented improvements in the quality of the human condition and technology is at the heart of this progress. Now we are experiencing an even bigger leap as we move towards a new level of digitalization and automation. Ranging from self-driving cars to factories without workers to societal infrastructure, every sensor and actuator is becoming connected and new applications that enable new opportunities are appearing daily. The fuel of this emerging connected, software-driven reality is software and the key challenge is to continuously deliver value to customers. The future of software engineering in this context is centered around a new, emerging digital business operating system consisting of four dimensions: Speed, Data, Ecosystems and Empowerment. The focus on speed is concerned with the constantly increasing rate of deploying new software in the field. This continuous integration and deployment is no longer only the purview of internet companies but is also increasingly deployed in embedded systems. Second, data is concerned with the vast amounts of information collected from systems deployed in the field and the behavior of the users of these systems. Software businesses need to significantly improve their ability to exploit the value present in that data. Third, ecosystems are concerned with the transition in many companies from doing everything in-house to strategic use of innovation partners and commodity providing partners. Finally, we need new ways of organizing work in this new, digital age. The keynote discusses these four main developments but focuses on the continuous software engineering. Also, the keynote provides numerous examples from the Nordic and international industry and predicts the next steps that industry and academia need to engage in to remain competitive.

Abstract Context: Continuous experimentation is frequently used in web-facing companies and it is starting to gain the attention of embedded systems companies. However, embedded systems companies have different challenges and requirements to run experiments in their systems. Objective: This paper explores the challenges during the adoption of continuous experimentation in embedded systems from both industry practice and academic research. It presents strategies, guidelines, and solutions to overcome each of the identified challenges. Method: This research was conducted in two parts. The first part is a literature review with the aim to analyze the challenges in adopting continuous experimentation from the research perspective. The second part is a multiple case study based on interviews and workshop sessions with five companies to understand the challenges from the industry perspective and how they are working to overcome them. Results: This study found a set of twelve challenges divided into three areas; technical, business, and organizational challenges and strategies grouped into three categories, architecture, data handling and development processes. Conclusions: The set of identified challenges are presented with a set of strategies, guidelines, and solutions. To the knowledge of the authors, this paper is the first to provide an extensive list of challenges and strategies for continuous experimentation in embedded systems. Moreover, this research points out open challenges and the need for new tools and novel solutions for the further development of experimentation in embedded systems.

Large software companies need to support continuous and fast delivery of customer value both in the short and long term. However, this can be hindered if both the evolution and maintenance of existing systems are hampered by Technical Debt. Although a lot of theoretical work on Technical Debt has been produced recently, its practical management lacks empirical studies. In this paper, we investigate the state of practice in several companies to understand what the cost of managing TD is, what tools are used to track TD, and how a tracking process is introduced in practice. We combined two phases: a survey involving 226 respondents from 15 organizations and an in-depth multiple case study in three organizations including 13 interviews and 79 Technical Debt issues. We selected the organizations where Technical Debt was better tracked in order to distill best practices. We found that the development time dedicated to managing Technical Debt is substantial (an average of 25% of the overall development), but mostly not systematic: only a few participants (26%) use a tool, and only 7.2% methodically track Technical Debt. We found that the most used and effective tools are currently backlogs and static analyzers. By studying the approaches in the companies participating in the case study, we report how companies start tracking Technical Debt and what the initial benefits and challenges are. Finally, we propose a Strategic Adoption Model for the introduction of tracking Technical Debt in software organizations.

Abstract Today, companies operate in business ecosystems where they collaborate, compete, share, and learn from others with benefits such as to present more attractive offerings and sharing innovation costs. With ecosystems being the new way of operating, the ability to strategically reposition oneself to increase or shift power balance is becoming key for competitive advantage. However, companies run into a number of traps when trying to realize strategical changes in their ecosystems. In this paper, we identify 5 traps that companies fall into. First, the “descriptive versus prescriptive trap” is when companies assume that current boundaries between partners are immutable. Second, the “assumptions trap” is when powerful ecosystem partners assume that they understand what others regard as value‐adding without validating their assumptions. Third, the “keeping it too simple trap” is when companies overlooks the effort required to align interests. Fourth, the “doing it all at once trap” is when companies disrupt an ecosystem assuming that all partners can change direction at the same time. Finally, the “planning trap” is when companies are unable to move forward without a complete plan. We provide empirical evidence for each trap, and we propose an ecosystem engagement process for how to avoid falling into these.

Abstract Companies need to know how much value their ideas deliver to customers. One of the most powerful ways to accurately measure this is by conducting online controlled experiments (OCEs). To run experiments, however, companies need to develop strong experimentation practices as well as align their organization and culture to experimentation. The main objective of this paper is to demonstrate how to run OCEs at large scale using the experience of companies that succeeded in scaling. Based on case study research at Microsoft, Booking.com, Skyscanner, and Intuit, we present our main contribution—The Experiment Growth Model. This four‐stage model addresses the seven critical aspects of experimentation and can help companies to transform their organizations into learning laboratories where new ideas can be tested with scientific accuracy. Ultimately, this should lead to better products and services.

Background: In the last two decades Technical Debt (TD) has received a considerable amount of attention from software engineering research and practice. Recently, a small group of studies suggests that, in addition to its technical and economic consequences, TD can affect developers' psychological states and morale. However, until now there has been a lack of empirical research clarifying such influences. Aims: In this study, we aim at taking the first step in filling this gap by investigating the potential impacts of TD and its management on developers' morale. Method: Drawing from previous literature on morale, we decided to explore the influence of TD and its management on three dimensions of morale called affective, future/goal, and interpersonal antecedents. In so doing, we conducted an exploratory field study and collected data from software professionals active in different industrial domains through eight qualitative interviews and an online survey (n=33). Results: Our results indicate that TD mainly has a negative influence on future/goal and affective antecedents of morale. This is mainly because the occurrence of TD hinders developers from performing their tasks and achieving their goals. TD management, on the other hand, has a positive influence on all the three dimensions of morale since it is associated with positive feelings and interpersonal feedback as well as a sense of progress. Conclusions: According to the results of this empirical study, the occurrence of TD reduces developers' morale, while its management increases developers' morale.

The negative consequences of Technical Debt is an area of increasing interest, and more specifically the Architectural aspects of it have received increased attention in the last few years. Besides the negative effects of Architectural Technical Debt on the overall software product quality in terms of hindering evolution and causing high maintenance costs, Architectural Technical Debt also has a significant negative impact on software practitioners' daily work. Although a great deal of theoretical work on Architectural Technical Debt has been undertaken, there is a lack of empirical studies that examine the negative effects of Architectural Technical Debt during the software development lifecycle. The aim of this study is to investigate how practitioners perceive and estimate the impact of Architectural Technical Debt during the software development process. This paper reports the results of an online web survey providing quantitative data from 258 participants. The contribution of this paper is threefold: First, it shows that practitioners experience that the Architectural type of Technical Debt has the highest negative impact on daily software development work. Secondly, we provide evidence that does not support the commonly held belief that Architectural Technical Debt increases with the age of the software. Thirdly, we show that despite different responsibilities and working tasks of software professionals, Architectural Technical Debt negatively affects all roles without any significant difference between the roles.

Online controlled experiments (for example A/B tests) are increasingly being performed to guide product development and accelerate innovation in online software product companies. The benefits of controlled experiments have been shown in many cases with incremental product improvement as the objective. In this paper, we demonstrate that the value of controlled experimentation at scale extends beyond this recognized scenario. Based on an exhaustive and collaborative case study in a large software-intensive company with highly developed experimentation culture, we inductively derive the benefits of controlled experimentation. The contribution of our paper is twofold. First, we present a comprehensive list of benefits and illustrate our findings with five case examples of controlled experiments conducted at Microsoft. Second, we provide guidance on how to achieve each of the benefits. With our work, we aim to provide practitioners in the online domain with knowledge on how to use controlled experimentation to maximize the benefits on the portfolio, product and team level.

Keeping the build fast is often stated as an important prerequisite for continuous integration, and is also widely discussed in literature. But which importance does the capacity of the build system actually have in relation to developer behavior? Based on metrics and interview results from a large-scale industry project, we present the factors that according to the developers themselves affect how often they deliver software to the mainline. We show that the developer will deliver less frequently if the delivery processes is time-consuming, if it's too complicated to deliver or if there is no evident value in delivering often to the mainline. Behind these three main themes, we also present a range of sub-categories such as architecture, test activities and administration. The build system capacity is one of several factors which, if not considered, could result in undesired continuous integration behaviors - but other factors should be seen as at least as important.

The EMFIS model allows companies to explicate a representation of the organization's current situation regarding continuous integration impediments, and visualizes what the organization must focus on in order to enable more frequent integration of software. The model is used to perform an assessment of twelve factors, where the ratings from participants representing the developers are summarized separately from ratings from participants representing the enablers (responsible for processes, development tools, test environments etc.). The EMFIS model is based on semi-structured interviews with 20 developers from two companies which develop large-scale software systems, and a literature review that included 74 research papers and four books. The model has been validated in workshops and interviews, which in total included 28 individuals in five different companies. The model was well received during the validation, and was appreciated for its simplicity and its ability to show differences in rating between developers and enablers.

Innovation and optimization in software systems can occur from pre-development to post-deployment stages. Companies are increasingly reporting the use of experiments with customers in their systems in the post-deployment stage. Experiments with customers and users are can lead to a significant learning and return-on-investment. Experiments are used for both validation of manual hypothesis testing and feature optimization, linked to business goals. Automated experimentation refers to having the system controlling and running the experiments, opposed to having the R&D organization in control. Currently, there are no systematic approaches that combine manual hypothesis validation and optimization in automated experiments. This paper presents concepts related to automated experimentation, as controlled experiments, machine learning and software architectures for adaptation. However, this paper focuses on how architectural aspects that can contribute to support automated experimentation. A case study using an autonomous system is used to demonstrate the developed initial architecture framework. The contributions of this paper are threefold. First, it identifies software architecture qualities to support automated experimentation. Second, it develops an initial architecture framework that supports automated experiments and validates the framework with an autonomous mobile robot. Third, it identifies key research challenges that need to be addressed to support further development of automated experimentation.

The importance of traceability in software development has long been recognized, not only for reasons of legality and certification, but also to enable the development itself. At the same time, organizations are known to struggle to live up to traceability requirements, and there is an identified lack of studies on traceability practices in the industry, not least in the area of tooling and infrastructure. This paper identifies traceability as a key challenge in achieving continuous integration and delivery and documents an industry developed framework - Eiffel - designed to provide real time traceability in continuous integration and delivery. The efficacy of the Eiffel framework is then investigated through comparison with previous traceability methods.

DevOps and continuous practices are attracting steadily growing attention by both practitioners and researchers in the software engineering community. The terms are often used inconsistently, interchangeably and with unclear meaning, however. Taking the position that this ambiguity and miscommunication renders the community great harm, in that it impedes our ability to critically appraise these practices, their effects and interplay between them, we analyze how published literature on both continuous practices and DevOps treat the terms. Based on this analysis, along with statements by often cited sources in the community as well as personal experience from researching and practicing these concepts, we propose guidelines to help authors reduce ambiguity in their publications. Additionally, definitions designed to reflect mainstream interpretation while disentangling the terms from one another are presented.

Based on interviews with 20 developers from two case study companies that develop large-scale software-intensive embedded systems, this paper presents the main factors that affect how often developers deliver software to the mainline. Further on, the paper describes the continuous integration behaviors in projects where up to 1,000 developers commit to the same mainline. The main factors that could enable more frequent integration of software are: "Activity planning and execution", "System thinking", "Speed" and "Confidence through test activities". Behind these main themes we also present a wide range of sub-categories ("Modular and loosely coupled architecture", "Test selection" etc) which summarizes what the developers themselves see as the continuous integration impediments in large-scale industry projects.

Blockchain is an emerging technology for decentralised and transactional data sharing across a large network of untrusted participants. It enables new forms of distributed software architectures, where agreement on shared states can be established without trusting a central integration point. A major difficulty for architects designing applications based on blockchain is that the technology has many configurations and variants. Since blockchains are at an early stage, there is little product data or reliable technology evaluation available to compare different blockchains. In this paper, we propose how to classify and compare blockchains and blockchain-based systems to assist with the design and assessment of their impact on software architectures. Our taxonomy captures major architectural characteristics of blockchains and the impact of their principal design decisions. This taxonomy is intended to help with important architectural considerations about the performance and quality attributes of blockchain-based systems.

Software development companies are increasingly aiming to become data-driven by trying to continuously experiment with the products used by their customers. Although familiar with the competitive edge that the A/B testing technology delivers, they seldom succeed in evolving and adopting the methodology. In this paper, and based on an exhaustive and collaborative case study research in a large software-intense company with highly developed experimentation culture, we present the evolution process of moving from ad-hoc customer data analysis towards continuous controlled experimentation at scale. Our main contribution is the "Experimentation Evolution Model" in which we detail three phases of evolution: technical, organizational and business evolution. With our contribution, we aim to provide guidance to practitioners on how to develop and scale continuous experimentation in software organizations with the purpose of becoming data-driven at scale.

Software companies need to support continuous and fast delivery of customer value both in short and a long-term perspective. However, this can be hindered by evolution limitations and high maintenance efforts due to internal software quality issues by what is described as Technical Debt. Although significant theoretical work has been undertaken to describe the negative effects of Technical Debt, these studies tend to have a weak empirical basis and often lack quantitative data. The aim of this study is to estimate wasted time, caused by the Technical Debt interest during the software life-cycle. This study also investigates how practitioners perceive and estimate the impact of the negative consequences due to Technical Debt during the software development process. This paper reports the results of both an online web-survey provided quantitative data from 258 participants and follow-up interviews with 32 industrial software practitioners. The importance and originality of this study contributes and provides novel insights into the research on Technical Debt by quantifying the perceived interest and the negative effects it has on the software development life-cycle. The findings show that on average, 36% of all development time is estimated to be wasted due to Technical Debt; Complex Architectural Design and Requirement Technical Debt generates most negative effect; and that most time is wasted on understanding and/or measuring the Technical Debt. Moreover, the analysis of the professional roles and the age of the software system in the survey revealed that different roles are affected differently and that the consequences of Technical Debt are also influenced by the age of the software system.

Abstract A known problem in large software companies is to balance the prioritization of short‐term and long‐term business goals. As an example, architecture suboptimality (Architectural Technical Debt), incurred to deliver fast, might hinder future feature development. However, some technical debt generates more interest to be paid than other. We conducted a multi‐phase, multiple‐case embedded case study comprehending 9 sites at 6 large international software companies. We have investigated which architectural technical debt items generate more interest , how the interest occurs during software development and which costly extra‐activities are triggered as a result. We presented a taxonomy of the most dangerous items identified during the qualitative investigation and a model of their effects that can be used for prioritization, for further investigation and as a quality model for extracting more precise and context‐specific metrics. We found that some architectural technical debt items are contagious, causing the interest to be not only fixed, but potentially compound, which leads to the hidden growth of interest (possibly exponential). We found important factors to be monitored to refactor the debt before it becomes too costly. Instances of these phenomena need to be identified and stopped before the development reaches a crises.

Editorial article Since the publication of Agile Manifesto in 2001, agile methods have transited from a grass- root initiative among enthusiastic advocates and developers to a mainstream software development approach adopted by both small and large companies worldwide. Meanwhile research on agile methods has grown rapidly and steadily into an established research area, evidenced by dedicated conferences (e.g., XP conference series, research track of previous Agile Conference series), special issues and sections in top Information Systems and Software Engineering journals. However, practitioners and consultants have largely driven the advancement in agile field, and agile research has lagged behind practice in the past. Has this situation changed as both agile methods and research community become increasingly mature? To be able to answer such questions, there is a constant need to check what interest agile practitioners and what agile researchers are investigating, to make sure that the states of the art and practice are aligned properly.

Recently, business ecosystems have been recognized as one of the most interesting phenomenon in software engineering research. Companies experience a paradigm shift where product development and innovation is moving outside the boundaries of the firm and where networks of stakeholders join forces to co-create value. While there is prominent research focusing on the managerial perspective of business ecosystems, few studies provide strategic guidance for how to intentionally manage the different ecosystems that companies operate in. Therefore, and on the basis of multicase study research, we provide empirical evidence on the challenges that software-intensive companies experience in relation to the different types of business ecosystems they operate in. We conduct a “state-of-the-art” literature review to identify strategies that are used to manage ecosystem engagements, and we develop a conceptual model in which we identify strategies for managing the innovation ecosystem, the differentiating ecosystem, and the commoditizing ecosystem. By categorising the different strategies in relation to the different types of ecosystems for which they are valid, the “three-layer ecosystem strategy model” provides comprehensive support for strategy selection. We validate the use of the identified strategies in 6 software-intensive case companies, and we provide empirical insights on the “relevance” and the “desired use” of these strategies as experienced by the case companies.

Large software companies need to support continuous and fast delivery of customer value both in the short and long term. However, this can be hindered if both evolution and maintenance of existing systems are hampered by Technical Debt. Although a lot of theoretical work on Technical Debt has been recently produced, its practical management lacks empirical studies. In this paper we investigate the state of practice in several companies in order to understand how they start tracking Technical Debt. We combined different methodologies: we conducted a survey, involving 226 respondents from 15 organizations and a more in-depth multiple case-study in three organizations, where Technical Debt was tracked: we involved 13 interviews and 79 Technical Debt issues analysis. We found that the development time dedicated to manage Technical Debt is substantial (around 25% of the overall development) but not systematic: only a few participants methodically track Technical Debt. By studying the approaches in the companies participating in the case-study, we understood how companies start tracking Technical Debt and what are the initial benefits and challenges. Finally, we propose a Strategic Adoption Model based to define and adopt a dedicated process for tracking Technical Debt.

Fast software deliveries are hindered by high maintenance efforts due to internal quality issues and Technical Debt (TD) and specifically, Architectural Technical Debt (ATD) has received increased attention in the last few years. ATD has a significant influence and impact on system success and, left unchecked, it can cause expensive repercussions, it is, therefore, of maintenance and evolutionary importance to understand the basic underlying factors of ATD. Thus, with this as background, there is a need for a descriptive model to illustrate and explain the different ATD issues. The aim of this study is to synthesize and compile research efforts with the goal of creating new knowledge with a specific interest in the ATD field. The contribution of this paper is the presentation of a novel descriptive model, providing a comprehensive interpretation of the ATD phenomenon. This model categorizes the main characteristics of ATD and reveals their corresponding relations. The model is based on a systematic literature review (SLR) of currently recognized knowledge concerning ATD.

With continuous deployment of software functionality, a constant flow of new features to products is enabled. Although new functionality has potential to deliver improvements and possibilities that were previously not available, it does not necessary generate business value. On the contrary, with fast and increasing system complexity that is associated with high operational costs, more waste than value risks to be created. Validating how much value a feature actually delivers, project how this value will change over time, and know when to remove the feature from the product are the challenges large software companies increasingly experience today. We propose and study the concept of a software feature lifecycle from a value point of view, i.e. how companies track feature value throughout the feature lifecycle. The contribution of this paper is a model that illustrates how to determine (1) when to add the feature to a product, (2) how to track and (3) project the value of the feature during the lifecycle, and how to (4) identify when a feature is obsolete and should be removed from the product.

Traditionally, innovation initiatives in software-intensive systems companies are viewed as either internal innovation, such as technology driven innovation based on ideas generated within a company, as collaborative innovation where a number of stakeholders co-create value, or as external innovation in which companies adopt strategies to capture and expand on ideas created by other stakeholders. However, and based on longitudinal case study research in six software-intense companies in the embedded systems domain, we see that most innovation strategies involve a mix of internal, collaborative and external elements. Due to the dichotomy in approaches however, companies often fail to select the optimal innovation strategy for the specific innovation challenge at hand. As a result, innovation initiatives suffer and companies and their ecosystem partners cannot fully capitalize on the value created. In this paper, we present a conceptual framework in which we identify twelve different ecosystem-centric innovation strategies. For each strategy, we identify the internal, the collaborative and the external elements. Also, and based on our empirical findings, we provide guidelines on the optimal selection of strategies.

The importance of traceability in software development has long been recognized, not only for reasons of legality and certification, but also to enable the development itself. At the same time, organizations are known to struggle to live up to traceability requirements, and there is an identified lack of studies on traceability practices in the industry, not least in the area of tooling and infrastructure. This paper identifies traceability as a key challenge in achieving continuous integration and delivery and documents an industry developed framework -- Eiffel -- designed to provide real time traceability in continuous integration and delivery. The efficacy of the Eiffel framework is then investigated through comparison with previous traceability methods.

As software development changes, also the myths and beliefs around it come and go. In different communities, different beliefs are kept, usually strengthened by success or failure stories. In this research, we study the beliefs surrounding software architecture. The beliefs range from the amount of effort needed for architecture documentation, to the size of the team or the persons responsible for making the architectural decisions. Most beliefs are based on the idea that the outcome of the project is highly dependent on the methods used during the design and development of software. We conducted a survey with 39 architects where we evaluated 54 architectural decisions. In this survey, we assessed the way in which decisions were made, the success factors of the decisions, as well as the properties of the projects. We conduct statistical analysis in order to evaluate some of the beliefs that currently exist in software development. We conclude that for most of the beliefs, no statistical evidence can be found, making these beliefs folklore for the tales, instead of useful guidelines for predicting projects success or failure.

The last century has seen unprecedented improvements in the quality of the human condition and technology is at the heart of this progress. Now we are experiencing an even bigger leap as we move towards a new level of digitization and automation. The fuel of this emerging Internet of Things reality is software and the key challenge is to continuously deliver value to customers. To deliver on this, organizations need to adopt a novel approach, evidence-based development, where data-driven decisions by R&D teams are complemented by automated experimentation by families of deployed systems. The keynote first introduces the aforementioned developments, then discusses the key areas of speed, data and ecosystems and finally presents the implications for organizations that seek to continuously deliver value to their customers.

The tutorial first discusses the trends that lead to our conclusion that the aforementioned factors are central for future software engineering. Based on this, we then discuss the implications that organizations will experience. These implications are concerned with the business, architecture, process and organization dimensions. For each dimension, we discuss specific methods and techniques. Examples include architecture technical debt management, visualization of continuous integration, techniques for evidence-based development as well as organizational models for combining architecture with agile development practices.

In order to continuously support the value delivery both in short-term and long-term, a key goal for large software companies is to continuously develop and manage software architecture. In order to understand how architecture management is employed in large Agile software companies, we have conducted interviews involving several roles at 5 firms. Through a combination of structured inductive and deductive analysis proper of Grounded Theory, we have identified current architect roles and gaps in the architecture practices in the studied organizations. From such investigation, we have developed an organizational framework, CAFFEA, for Agile architecting, including roles, (virtual) teams and practices. The framework has been evaluated through a cross-company workshop including participants from 5 large software companies, discussion groups and a final survey. Finally, we have evaluated the framework in practice after one year of its application at one of the companies. We found that some necessary architectural practices are overlooked in Large Agile Software Development. The evaluation of CAFFEA framework showed that the included roles and teams are needed in order to mitigate the gaps in the architectural practices. The responsibilities and the activities have been mapped to key architect roles compatible with the Scrum setting employed at the companies. The evaluation of CAFFEA shows key benefits.

With agile teams becoming increasingly multi-disciplinary and including all functions, the role of customer feedback is gaining momentum. Today, companies collect feedback directly from customers, as well as indirectly from their products. As a result, companies face a situation in which the amount of data from which they can learn about their customers is larger than ever before. In previous studies, the collection of data is often identified as challenging. However, and as illustrated in our research, the challenge is not the collection of data but rather how to share this data among people in order to make effective use of it. In this paper, and based on case study research in three large software-intensive companies, we (1) provide empirical evidence that ‘lack of sharing’ is the primary reason for insufficient use of customer and product data, and (2) develop a model in which we identify what data is collected, by whom data is collected and in what development phases it is used. In particular, the model depicts critical hand-overs where certain types of data get lost, as well as the implications associated with this. We conclude that companies benefit from a very limited part of the data they collect, and that lack of sharing of data drives inaccurate assumptions of what constitutes customer value.

Many consumer electronics firms are adopting an ecosystem-centric approach for supporting third-party applications. In an emerging market, a consumer electronics firm may need to create a new ecosystem or adopt a newly developed platform, both which has significant commercial and technical implications. In this paper we identify three types of ecosystems that are used today: vertically integrated hardware/software platforms, closed-source software platforms, and open-source software platforms. We introduce a first step towards a multi-criteria decision support method, which determines what type of ecosystem is most suitable for a specific product category from a software engineering perspective. We use this method to analyze a wide range of consumer electronics products. The analysis shows that the vertically integrated platform type is most suitable for product with a high degree of innovation, and open-source software platforms are more suitable when a large amount of variants are needed. The closed-source software platform type is less suitable for most types of consumer electronics devices. This first step towards a full-fledged decision support method can be used by platform owners to decide whether their platform can also be successfully transferred to another product type.

The adoption of Agile Software Development in large companies is a recent phenomenon of great interest both for researchers and practitioners. Although intra-team interaction is well supported by established agile practices, the critical interaction between the agile team and other parts of the organization is still unexplored in literature. Such interactions slow down the development, hindering the achievement of business goals based on speed: short time to market, quick replication of products of a product-line, and reaction time for product evolution. We have employed a two-year long multiple-case case-study, collecting data through interviews and a survey in three large companies developing embedded software. Through a combination of qualitative and quantitative analysis, we have found strong evidence that interaction challenges between the development team and other groups in the organization hinder speed and are widespread in the organizations. This paper also identifies current practices in use at the studied companies and provides detailed guidelines for novel solutions in the investigated domain. Such practices are called boundary-spanning activities in information system research and coordination theory. We present a comparison between large embedded software companies employing agile and developing a line of products based on reused assets and agile companies developing pure software. We highlight specific contextual factors and areas where novel spanning activities are needed for mitigating the interaction challenges hindering speed. Copyright © 2015 John Wiley & Sons, Ltd.

Continuous software engineering refers to the organizational capability to develop, release and learn from software in very short rapid cycles, typically hours, days or a very small numbers of weeks. This requires not only agile processes in teams but in the complete research and development organization. Additionally, the technology used in the different development phases, like requirements engineering and system integration, must support the quick development cycles. Finally, automatic live experimentation for different system alternatives enables fast gathering of required data for decision making. The workshop, the second in the series after the first one at ICSE 2014, aims to bring the research communities of the aforementioned areas together to exchange challenges, ideas, and solutions to bring software engineering a step further to being a holistic continuous process. The workshop program is based on eight papers selected in the peer-review process and supplemented by interaction and discussions at the workshop. The topics range from agile methods, continuous software engineering practices to specific techniques, like visualization and testing.

The automotive industry adopts various approaches to reduce the production lead time in order to be competitive on the market. Due to the increasing amount of in-house software development, this industry gets new opportunities to decrease the software development lead-time. This can have a significant impact on decreasing time to market and fewer resources spent in projects. In this paper we present a study of software development areas where we perceived barriers for fast development and where we have identified enablers to overcome these barriers. We conducted a case study at one of the vehicle manufacturers in Sweden using structured interviews. Our results show that there are 21 barriers and 21 corresponding enablers spread over almost all phases of software development.

A known problem in large software companies is to balance the prioritization of short-term with long-term viability. Specifically, architecture violations (Architecture Technical Debt) taken to deliver fast might hinder future feature development. However, some technical debt requires more interest to be paid than other. We have investigated which Technical Debt items generate more effort and how this effort is manifested during software development. We conducted a multiple-case embedded case study comprehending 7 sites at 5 large international software companies. We found that some Technical Debt items are contagious, causing other parts of the system to be contaminated with the same problem, which may lead to non-linear growth of interest. We also identify another socio-technical phenomenon, for which a combination of weak awareness of debt, time pressure and refactoring creates Vicious Circles of events during the development. Such phenomena need to be identified and stopped before the development is led to a crisis point. Finally, this paper presents a taxonomy of the most dangerous items identified during the qualitative investigation and a model of their effects that can be used for prioritization, for further investigation and as a quality model for extracting more precise and context-specific metrics.

With the increasing rate of change in industry, companies need to manage their internal alignment between strategy, software architecture and organizing R&D as well as with their ecosystem more proactively. Lack of alignment is the root cause of many problems experienced in the software-intensive systems industry. The ESAO model provides a framework for addressing the aforementioned challenges. It provides an internal perspective as well as an ecosystem perspective and for each perspective offers a strategic, architectural and ways of working dimension. ESAO offers tools for each of the dimensions as well as mechanisms for establishing and maintaining alignment.

The variability of product lines increases over time thereby leading to an increasing effort for testing. Since the available time for test activities is limited an efficiency improvement is needed to ensure that products have sufficient quality.

An open issue in industry is the combination of software reuse in the context of large scale Agile Software Development. The speed offered by Agile Software Development is needed for short time to market, while reuse strategies such as Software Product Line Engineering are needed for long-term productivity, efficiency, and profit. The paper investigates, through a survey, communication factors affecting both speed and reuse in 3 large companies developing embedded systems and employing Agile Software Development and Software Product Line Engineering. Our results include a prioritized list of communication related factors obtained by statistical analysis and the recognition and spread of the factors in the companies. We have recognized 5 interfaces with the Agile development team that need to be improved: system engineers (architects), product management, distributed teams, inter-project teams and sales unit. Few factors (involving inter-project communication) depend on the business drivers for the company. We also reveal that Agile teams need strategic and architectural inputs in order to be implanted in a large company employing Software Product Line Engineering. Academic and industrial training as well as different tactics for co-location would improve the communication skills of engineers. There is also a need for solutions, in the reference architecture, for fostering Agile Software Development: the goal is the combination of the focus on customer value of the teams, reusability, system requirements and avoidance of organizational dependencies.

Since its rise to general awareness and popularity starting close to two decades ago, the concept of software product lines has taken the center stage in the software reuse community. After more than four decades of research into effective and efficient reuse of software inside the four walls of the organization, and countless initiatives, software product lines presented an approach that has proven to provide real productivity improvements in the development cost of software intensive products. This has allowed companies to increase their product portfolio with an order of magnitude, to allow for much higher degrees of configurability by customers, facilitated common look-and-feel across a wide product population and enabled companies to be more innovative by decreasing the cost of new product experiments. It achieved this by broadening the scope of study from technology to include process, business strategy and organizational aspects. Successful product lines address all aspects relevant to the organization and then adopt and institutionalize the approach in the company. This special issue features eight papers from the 14th Software Product Line Conference (SPLC 2010). SPLC provides an institution and the premier meeting place for the software product line community. In particular, SPLC 2010 was held in Jeju, South Korea, and the accepted papers covered various areas of software product line engineering including product line contexts, variability management, formal approaches, product validation, and feature modeling. We invited eight top quality papers and they were significantly improved and extended for this special issue.

An open issue in industry is software reuse in the context of large scale Agile product development. The speed offered by agile practices is needed to hit the market, while reuse is needed for long-term productivity, efficiency, and profit. The paper presents an empirical investigation of factors influencing speed and reuse in three large product developing organizations seeking to implement Agile practices. The paper identifies, through a multiple case study with 3 organizations, 114 business-, process-, organizational-, architecture-, knowledge- and communication factors with positive or negative influences on reuse, speed or both. Contributions are a categorized inventory of influencing factors, a display for organizing factors for the purpose of process improvement work, and a list of key improvement areas to address when implementing reuse in organizations striving to become more Agile. Categories identified include good factors with positive influences on reuse or speed, harmful factors with negative influences, and complex factors involving inverse or ambiguous relationships. Key improvement areas in the studied organizations are intra-organizational communication practices, reuse awareness and practices, architectural integration and variability management. Results are intended to support process improvement work in the direction of Agile product development. Feedback on results from the studied organizations has been that the inventory captures current situations, and is useful for software process improvement work.

Runtime variability is a key technique for the success of Dynamic Software Product Lines (DSPLs), as certain application demand reconfiguration of system features and execution plans at runtime. In this emerging research work we address the problem of dynamic changes in feature models in sensor networks product families, where nodes of the network demand dynamic reconfiguration at post-deployment time.

Software product line companies increasingly expand their platform outside their organizational boundaries, in effect transitioning to a software ecosystem approach. In this tutorial, we discuss the emerging trend of software ecosystems and provide a overview of the key concepts and implications of adopting a software ecosystem approach. We define the notion of software ecosystems and introduce a taxonomy. Finally, we explore the implications of software ecosystems to the way companies build software.

The ability to rapidly respond to customer interest and to effectively prioritize development effort has been a longstanding challenge for mass-market software intensive products. This problem is exacerbated in the context of software product lines as functionality may easily fall over software asset and organizational boundaries with consequent losses in efficiency and nimbleness. In this tutorial, we discuss these challenges; present a case study, Intuit's Quickbooks product line that combined agile software development, design thinking and self-organizing teams in a successful approach; and discuss a generalized approach and implications from employing the techniques in your organization.

Abstract The ability to rapidly respond to customer interest and to effectively prioritize development effort has been a long‐standing challenge for mass‐market software intensive products. This problem is exacerbated in the context of software product lines as functionality may easily fall over software asset and organizational boundaries with consequent losses in efficiency and nimbleness. Some companies facing these problems in their product line respond with a new development process. In this paper we discuss the developments within a single case study, Intuit's Quickbooks product line that combined agile software development, design thinking and self‐organizing teams in a successful approach, which provided a significant improvement in terms of responsiveness and accuracy of building customer value. Copyright © 2011 John Wiley & Sons, Ltd.

Large-scale software development, for the longest time, has relied heavily on centralized, process-centric approaches, such as CCMI. There are three trends that make applying a traditional approach increasingly infeasible, i.e. the increasing adoption of software product lines, global software development and software ecosystem.

Abstract This article compares the organization and practices for software reuse in integration‐oriented software product lines (SPLs) and open source software projects. The main observation is that both approaches are successful regarding large variability and reuse, but differ widely in their practices and organization. To capture practices in large open source projects, we describe an open compositional model, which reflects their more decentralized organization of software development. We capture key practices and organizational forms for this and validate these by comparing four case studies of this model. Two of these studies are based on published SPL case studies, for the other two we analyze the practices in two large and successful open source projects based on their published developer documentation. Our analysis highlights key differences between the practices in the two open source organizations and the more integrational practices used in the other two cases. Finally, we discuss which practices are successful in which environment and how the current practices can move towards more open, widely scoped and distributed software development. Copyright © 2010 John Wiley & Sons, Ltd.

Provides notice of upcoming conference events of interest to practitioners and researchers.

The earliest design decisions often have a significant impact on software quality and are the most costly to revoke. One of the challenges in architecture design is to reduce the frequency of retrofit problems in software designs; not being able to improve the quality of a system cost effectively, a problem which frequently occurs during late stages. Software architecture assessment is essential in the design of a high quality system. However, assessing the effect of individual design decisions with respect to quality is often complicated by the fact that it is hard to identify exactly how particular qualities and quality factors are improved or impaired by design decisions. In this paper we present a framework that formalizes some of the relationships between software architecture and software quality; it compiles existing design knowledge (quality improving patterns) in a format suitable for architecture assessment. This framework may prevent the retrofit problem and can assist in reasoning about intra- and inter- quality tradeoffs. We illustrate our framework by creating an instance for it for the qualities usability, security and safety.

Software architectures have high costs for change, are complex, and erode during evolution. We believe these problems are partially due to knowledge vaporization. Currently, almost all the knowledge and information about the design decisions the architecture is based on are implicitly embedded in the architecture, but lack a first-class representation. Consequently, knowledge about these design decisions disappears into the architecture, which leads to the aforementioned problems. In this paper, a new perspective on software architecture is presented, which views software architecture as a composition of a set of explicit design decisions. This perspective makes architectural design decisions an explicit part of a software architecture. Consequently, knowledge vaporization is reduced, thereby alleviating some of the fundamental problems of software architecture.

WICSA is a working conference and in 2005 continued the tradition of providing working sessions as a forum supportive of communication among the diverse population that makes up the software architecture community. The working sessions of WICSA 5 reflected on the achievements in the field of software architecture during the past decade, took stock of the current state of the field, and identified opportunities for researchers to contribute to improving the practice.

We present the results of two case studies we conducted at Baan in the Netherlands. At the time of conducting the case studies, Baan was part of Invensys plc. (Baan is now owned by SSA Global Technologies.) In these case studies we investigated how companies identify design erosion and address this in their software, a practice we call ‘design preservation’. In this study, we selected two sub-systems in Baan products that had recently been subjected to extensive maintenance activities because they were eroded. In this paper, we analyze the problems these systems had, how Baan identified that these systems were problematic, and the remedies that were used to address the problems. In addition to confirming some of our earlier conclusions, we have been able to extract some common causes for design erosion problems as well as a number of recommended design preservation practices, which, at least for Baan, have proven to be very effective in strengthening design preservation. Copyright © 2005 John Wiley & Sons, Ltd.

Development of software product families relies heavily on the use of variability to manage the differences between products by delaying design decisions to later stages of the development and usage of the constructed software systems. Implementation of variability is not a trivial task, and is governed by a number of factors. In this paper, we describe the factors that are relevant in determining how to implement variability, and present a taxonomy of variability realization techniques. Copyright © 2005 John Wiley & Sons, Ltd.

In a software product family context, software architects design architectures that support product diversification in both space (multiple contexts) and time (changing contexts). Product diversification is based on the concept of variability: a single architecture and a set of components support a family of products. Software product families have to support increasing amounts of variability, but variability dependencies are often ill-defined and have unexpected or even unknown behavior This paper suggests a (1) taxonomy and a (2) hierarchy of variability dependencies in a software product family context. The taxonomy is based on the concept of variation points and identifies four main types of variability dependencies. The four-tier hierarchy is a generalization of the architectural layers found in the so-called building block method

During recent years, the amount of variability that has to be supported by a software artefact is growing considerably and its management is evolving into a major challenge during development, usage, and evolution of software artefacts. Successful management of variability in software leads to better customizable software products that are in turn likely to result in higher market success. The aim of this paper is to present software variability management both from a 'problems' and from a 'solutions' perspective by discussing experiences from industrial practice and from applied research in academia. Issues that are addressed include, but are not limited to, technological, process, and organizational aspects as well as notation, assessment, design, and evolution aspects.

Evolution of variability is a key factor in the successful exploitation of commonalities in software product families. Assessment of variability can be used to determine how the variability provided by a product family should evolve. We present COSVAM (COVAMOF software variability assessment method), a variability assessment technique that specifically addresses evolution of variability. We exemplify our approach with the Dacolian case study.

In a software product family context, software architects design architectures that support product diversification in both space (multiple contexts) and time (changing contexts). Product diversification is based on the concept of variability: a single architecture and a set of components support a family of products. Software product families have to support increasing amounts of variability, thereby making variability engineering a primary concern in software product family development. The first part of this paper (1) suggests a two-dimensional, orthogonal categorization of variability realization techniques and classifies these variability categories into system maturity levels. The second part (2) discusses a case study of an industrial software product family of mobile communication infrastructure for professional markets such as the military. The study categorizes and classifies the variability in this product family according to criteria common to virtually all software development projects.

Abstract Promoting software reuse is probably the most promising approach to the cost‐effective development and evolution of quality software. An example of reuse is the successful adoption of software product families in industry. In a product family context, software architects anticipate product variation and design architectures that support product derivation in both space (multiple contexts) and time (changing contexts). Product derivation is based on the concept of variability: a single architecture and a set of components support a family of products. Modern software product families need to support increasing amounts of variability, leading to a situation where variability engineering becomes of primary concern. Variability is often introduced as an ‘add‐on’ to the system without taking the consequences for more than one lifecycle phase such as design or architecture into account. This paper (1) suggests a Variability Categorization and Classification Model (VCCM) for representing variability in the software lifecycle and (2) discusses a case study of a large‐scale software product family of Magnetic Resonance Imaging (MRI) scanners developed by Philips Medical Systems. The study illustrates how variability can be made an integral part of system development at different levels of abstraction. VCCM has been applied to the scanner family as an analysis tool. Copyright © 2003 John Wiley & Sons, Ltd.

Usability is increasingly recognized as a quality attribute that one has to design for. The conventional alternative is to measure usability on a finished system and improve it. The disadvantage of this approach is, obviously, that the cost associated with implementing usability improvements in a fully implemented system are typically very high and prohibit improvements with architectural impact. In this tutorial, we present the insights gained, techniques developed and lessons learned in the EU-IST project STATUS (SofTware Architectures That supports USability). These include a forward-engineering perspective on usability, a technique for specifying usability requirements, a method for assessing software architectures for usability and, finally, for improving software architectures for usability. The topics are extensively illustrated by examples and experiences from many industrial cases.

The First International Workshop on the Relationships between Software Engineering and Human-Computer Interaction was held on May 3-4, 2003 as part of the 2003 International Conference on Software Engineering, in Portland, OR, U.S.A. This workshop was motivated by a perception among researchers, practitioners, and educators that the fields of Human-Computer Interaction and Software Engineering were largely ignoring each other and that they needed to work together more closely and to understand each other better. This paper describes the motivation, goals, organization, and outputs of the workshop.

During recent years, the amount of variability that has to be supported by a software artifact is growing considerably and its management is developing as a main challenge during development, usage, and evolution of software artifacts. Successful management of variability in software artifacts leads to better customizable software products that are in turn likely to result in higher market success. The aim of this workshop is to study software variability management both from a 'problems' and from a 'solutions' perspective by bringing together people from industrial practice and from applied research in academia to present and discuss their respective experience. Issues to be addressed include, but are not limited to, technological, process, and organizational aspects as well as notation, assessment, design, and evolution aspects.

No abstract available.

Software product lines are enjoying increasingly wide adoption in the software industry. Most authors focus on the technical and process aspects, and assume an organizational model consisting of a domain engineering unit and several application engineering units. In our cooperation with several software development organizations applying software product-line principles, we have identified several other organizational models that are employed as well. In this article, we present a number of organizational alternatives, organized around four main models, viz. a development department, business units, domain engineering unit and hierarchical domain engineering units. For each model, its characteristics, applicability, advantages and disadvantages are discussed, as well as an example. Based on an analysis of these models, we present three factors that influence the choice of the organizational model, viz. product-line assets, responsibility levels and the type of organizational units.

The authors discuss the notion of variability. We have experienced that this concept has so far been underdefined, although we have observed that variability techniques become increasingly important. A clear indication of this trend is the recent emergence of software product lines. Software product lines are large, industrial software systems intended to specialize into specific software products. The authors provide a framework of terminology and concepts regarding variability. In addition, they present three recurring patterns of variability. Finally, they suggest a method for managing variability in software product lines.

Abstract Object‐oriented frameworks provide software developers with the means to build an infrastructure for their applications. Unfortunately, frameworks do not always deliver on their promises of reusability and flexibility. To address this, we have developed a conceptual model for frameworks and a set of guidelines to build object oriented frameworks that adhere to this model. Our guidelines focus on improving the flexibility, reusability and usability (i.e. making it easy to use a framework) of frameworks. Copyright © 2001 John Wiley & Sons, Ltd.

Object-oriented framework technology has become a common reuse technology in software development. As with all software, frameworks evolve over time. Once the framework has been deployed, new versions of a framework potentially cause a high maintenance cost for the products built with the framework. This fact, in combination with the high costs of developing and evolving a framework, make it important for organizations to achieve a controlled and predictable evolution of the framework's functionality and costs. We present a metrics-based framework stability assessment method, which has been applied on two industrial frameworks from the telecommunication and graphical user interface domains. First, we discuss the framework concept and the frameworks studied. Then, the stability assessment method is presented including the metrics used. The results from applying the method, as well as an analysis of each of the frameworks, are described. We continue with a set of observations regarding the method, including framework differences that seem to be invariant with respect to the method. A set of framework stability indicators based on the results is then presented. Finally, we assess the method against issues related to the management and evolution of frameworks, framework deployment, change impact analysis and benchmarking. Copyright © 2000 John Wiley & Sons, Ltd.

A challenge in software design is to find solutions that balance and optimize the quality attributes of the application. We present a case study of an application and the results of a design decision made on weak assumptions. The application has been assessed with respect to performance and maintainability. We present and evaluate an alternative design of a critical system component. Based on interviews with the involved designers we establish the design rationale. By analyzing the evaluation data of the two alternatives and the design rationale, we conclude that the design decision was based on a general assumption that an adaptable component design should increase the maintainability of the application. This case study is clearly a counter example to that assumption, and we therefore reject it as a myth. This study shows, however, that the myth is indeed responsible for the major performance problem in the application.

In this paper we present the experiences and architecture from a research project conducted in cooperation with two industry partners.The goal of the project was to reengineer an existing system for haemo dialysis machines into a domain specific software architecture [23].Our main experiences are (1) architecture design is an iterative and incremental process, (2) software qualities require a context, (3) quality attribute assessment methods are too detailed for use during architectural design, (4) application domain concepts are not the best abstractions, (5) aesthetics guides the architect in finding potential weaknesses in the architecture, ( 6) it is extremely hard to decide when an architecture design is ready, and (7) documenting software architectures is a important problem.We also present the architecture and the design rational to give a basis for our experiences.We evaluated the resulting architecture by implementing a prototype application.

In this paper, a case study investigating the experiences from using product-line architectures is presented involving two Swedish companies, Axis Communications AB and Securitas Larm AB.Key persons in these organizations have been interviewed and information has been collected from documents and other sources.The study identified a collection of problems and issues.The identified problems include the amount of required background knowledge, information distribution, the need for multiple versions of assets, dependencies between assets, use of assets in new contexts, documentation, tool support, management support and effort estimation.Issues collected from the case study are the questioned necessity of domain engineering units, business units versus development departments, time-tomarket versus asset quality and common features versus feature superset.For each problem, a problem description, an example, underlying causes, available solutions and research issues are identified whereas for each issue the advantages and disadvantages of each side are discussed.

An approach for identifying potential modules for restructuring in large software systems using product release history is adapted to better suit object-oriented frameworks and is applied to an industrial black-box framework product in the telecommunication domain. Our study shows that using historical information as a way of identifying structural shortcomings in an object-oriented system is viable and useful. The study thereby strengthens the approach and demonstrates that the approach is adaptable and useful for object-oriented systems. The usefulness of the original approach has been validated through this study too.

This paper discuss the results of two case studies from a technical perspective, concentrating on the evolution of software assets in two Swedish organizations that have employed a product-line architecture approach for several years. This paper describes and analyses the commonalities and differences of these two cases, emphasising categories of the evolution of the requirements, of the software architecture and of the software components. This paper concludes with three types of lessons learned about evolution in software product lines: three evolution categories are predominant, three other categories are less significant but still common, and seven guidelines for software product-line evolution emerge. Copyright © 1999 John Wiley & Sons, Ltd.

The paper presents a method for reengineering software architectures. The method explicitly addresses the quality attributes of the software architecture. Assessment of quality attributes is performed primarily using scenarios. Design transformations are done to improve quality attributes that do not satisfy the requirements. Assessment and design transformation can be performed for several iterations until all requirements are met. To illustrate the method we use the reengineering of a prototypical measurement system into a domain-specific software architecture as an example.

Conventional compilers often are large entities that are highly complex, difficult to maintain and hard to reuse. In this article it is argued that this is due to the inherently functional approach to compiler construction. An alternative approach to compiler construction is proposed, based on object-oriented principles, which solves (or at least lessens) the problems of compiler construction. The approach is based on delegating compiler objects ( Dco s) that provide a structural decomposition of compilers in addition to the conventional functional decomposition. The DCO approach makes use of the parser delegation and lexer delegation techniques, that provide reuse and modularisation of syntactical, respectively, lexical specifications. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

When modelling complex computer systems, four primary modelling techniques are available to the software engineer, i.e. abstraction, composition, functional decomposition and multiple client interfaces (views). Abstraction and composition can directly be expressed in traditional object oriented models and functional decomposition can often be expressed using subsystems. However, traditional object models provide no expressiveness for multiple client interfaces. This forces designers to either model all different views on a conceptual object as a large, complex implementation object or to model each view as an object and relate the different view objects. Both approaches suffer from problems of complexity, maintainability and reusability. As a solution, we propose the layered object model that allows for the expressive, flexible and extensible definition of multiple client interfaces. To illustrate the problems associated with views and our proposed solution, the domain of industrial control systems is used.