Platform ecosystems have revolutionized value creation across numerous industries, inducing technical leaps and scaling artifact generation in scales and speeds unattainable through traditional vertically integrated or single-firm models. These ecosystems rely on collaborative interactions among actors to co-create and reuse value. Such ecosystems are socio-technical environments which require complex governance and orchestration strategies to ensure ecosystem health and performance from financial, technical, and social perspectives. Consequently, monitoring the performance of platform ecosystems requires non-primitive metrics as factors contributing to ecosystem performance are multifaceted compared to conventional software settings. Effective orchestration of platform ecosystems requires relies on access to real-time quantitative performance indicators. The existing literature offers various quantitative health metrics and performance indicators for platform ecosystems, yet these are dispersed across multiple studies and often embedded in abstract models or found within generic analytics systems. This research reviews existing quantitative real-time health metrics and performance indicators of platform ecosystems. We identified 417 distinct metrics after eliminating duplicates and incomplete definitions, and refined 168 of these metrics to be calculable in real-time using a consistent framework for definition, nomenclature, and quantification. Furthermore, we compiled existing real-time ecosystem health monitoring methods into a reference architecture and tested its feasibility in two active platform ecosystems. The study yields four key contributions: a practical catalog of platform ecosystem health metrics; a reference architecture for creating real-time ecosystem health monitoring solutions, demonstrated through implementation in two operational platform ecosystems; industry-relevant insights for practitioners; and a discussion of potential future research directions.

Context: Software ecosystems have transformed many industries, redefining collaboration and value co-creation. The success of such ecosystems depends on the dynamism of the network of users on its different sides. Consequently, decision-making in such multifaceted and interconnected environments is more complex than in conventional software products. On-line controlled experiments are considered the gold standard for aiding decision-making in software engineering processes. Experiments are extensively used to reduce bias and estimation noise for design, engineering, and business decisions. However, experimentation in software ecosystems is inherently more com-plex as it deals with atypical sources of bias and technical complications. Primary studies of experimentation approaches in software ecosystems are scattered across multiple domains and disciplines, and secondary research on the topic is scarce as highlighted in different tertiary studies. Hence, we conducted this study. Objectives: To explore primary research on experimentation in software ecosystems; Summarize current approaches, toolboxes, and solutions that practitioners and researchers, facing similar problems, can use to inform their approaches; To outline underexplored research areas and provide recommendations for practitioners. Method: We conducted a systematic literature review. The search strategy, application of exclusion and inclusion criteria, and subsequent quality assessment resulted in 63 relevant studies. Data extraction process was designed and carried out to collect data relevant to the study objectives. The extracted data under-went descriptive and thematic syntheses and analyses, in addition to cross-analysis on relevant axes. Contributions: The study resulted in four contributions. First, a distillation of the themes and patterns in the available research on the topic. Second, a practical summary of the experimental designs specific to each software ecosystem type. Third, an actionable road map for practitioners in order to achieve exper-imentation maturity in software ecosystems. Fourth, an outline of the underexplored research areas.

Industrial software ecosystems refer to a network of interdependent actors, co-creating value through a shared technological platform specifically tailored to industrial sectors. Developing, maintaining, and orchestrating such platforms involves many challenges that require complex decision making. Experimentation can help alleviate this complexity and reduce decision uncertainty and bias. However, experimentation requires certain organizational, infrastructural, and data-related prerequisites which can be uniquely challenging to achieve in industrial software ecosystems. Through semi-structured interviews with 25 industry professionals involved in various roles across 17 ecosystems, we analyze the difficulties faced in conducting effective experiments in such environments. The interview protocol covered aspects related to the methodologies, data handling processes, and current experimentation practices, as well as the challenges faced by practitioners who engage in experimentation initiatives. The study findings reveal technical, organizational, and market-related challenges, detailing the complexities facing experimentation initiatives in industrial software ecosystems. The findings are presented in an actionable manner, following a model that allows business-oriented alignment of architecture, process, and organizational evolution strategies. The study identifies key impediments, such as data integration difficulties, stringent regulatory environments, and prevailing organizational cultures that hinder continuous experimentation practices. Our analysis provides a foundation for understanding the unique challenges facing experimentation efforts in industrial software ecosystems and offers insights into potential strategies to improve the effectiveness of these initiatives.

The emergence of platform ecosystems has transformed the business landscape in many industries, giving rise to novel modes of interorganizational cooperation and value co-creation, as well as unconventional challenges. The vast traces of data generated by platform ecosystems makes them ripe for the use of analytics and data-driven methods aimed at improving their health, performance, business outcomes, and evolution. However, the research on the application of analytics within platform ecosystems is limited and spread across multiple disciplines. To address this gap, we conducted a systematic literature review on the application of analytics and data-driven methods and practices within platform ecosystems. A total of 56 studies were reviewed, and underwent data extraction, analysis, and synthesis processes. In addition to presenting themes and patterns in the recent and relevant literature on platform ecosystems analytics, our review offers the following outcomes: an actionable overview of the analytics toolbox currently used within platform ecosystems—spanning domains such as machine learning, deep learning, data science, modelling, simulation, among others—; a roadmap for practitioners to achieve analytics maturity; and a summary of underexplored research areas.

In order to distinguish themselves from their competitors, software service providers constantly try to assess and improve customer satisfaction. However, measuring customer satisfaction in a continuous way is often time and cost intensive, or requires effort on the customer side. Especially in B2B contexts, a continuous assessment of customer satisfaction is difficult to achieve due to potential restrictions and complex provider-customer-end user setups. While concepts such as web usage mining enable software providers to get a deep understanding of how their products are used, its application to quantitatively measure customer satisfaction has not yet been studied in greater detail. For that reason, our study aims at combining existing knowledge on customer satisfaction, web usage mining, and B2B service characteristics to derive a model that enables an automated calculation of quantitative customer satisfaction scores. We apply web usage mining to validate these scores and to compare the usage behavior of satisfied and dissatisfied customers. This approach is based on domain-specific service quality and web usage metrics and is, therefore, suitable for continuous measurements without requiring active customer participation. The applicability of the model is validated by instantiating it in a real-world B2B online platform.

In recent years, the application of artificial intelligence (AI) has become an integral part of a wide range of areas, including software engineering. By analyzing various data sources generated in software engineering, it can provide valuable insights into customer behavior, product performance, bugs and errors, and many more. In practice, however, AI for software analytics and business intelligence often gets stuck in a prototypical stage and the results are rarely used to make decisions based on data. To understand the underlying root causes of this phenomenon, we conduct both an explanatory case study and a survey on the challenges of realizing and utilizing artificial intelligence in the context of software-intensive businesses. As a result, we identify a vicious circle that prevents practitioners from moving from prototypical analytics to continuous and productively usable software analytics and business intelligence based on AI.

A wide variety of tools to monitor and track software systems, such as websites or smartphone applications, during runtime already exists. However, their aggregated results are often not sufficient to answer questions on a product management level since these questions address several levels of complexity and abstractions, and tend to be formulated on a rather high level, for instance concerning the efficiency of their website structure for their users. A straightforward mapping between low level metrics and high level insights is typically not possible. This causes a gap that makes it challenging to continuously provide quantitative high-level insights in real-time. In order to address this challenge, we conducted a study within three distinct platforms and products, and propose a model based on our results. After defining a case for each of the independent platforms and products, we implemented a process to measure high level insights using low level metrics for each of these cases. Next, we compared the procedures and steps that were taken in each of the cases and derived a model that describes a generic approach how to utilize and process data in order to gain higher level insights. Our model structures the steps from data to knowledge over different levels of complexity and abstraction, namely operational, tactical, and strategic. Thereby, the knowledge acquired in each phase serves as input in the next phase which increases the measurable level of complexity with each iteration. Since the steps in our model are specifically arranged as a pipeline, it enables practitioners to automate a continuous and quantitative measurement of high level insights in real-time.

Abstract The shift from sequential to agile software development originates from relatively small and co-located teams but soon gained prominence in larger organizations. How to apply and scale agile practices to fit the needs of larger projects has been studied to quite an extent in previous research. However, scaling agile beyond organizational boundaries, for instance in a software ecosystem context, raises additional challenges that existing studies and approaches do not yet investigate or address in great detail. For that reason, we conducted a case study in two software ecosystems that comprise several agile actors from different organizations and, thereby, scale development across organizational boundaries, in order to elaborate and understand their coordination challenges. Our results indicate that most of the identified challenges are caused by long communication paths and a lack of established processes to facilitate these paths. As a result, the participants in our study, among others, experience insufficient responsivity, insufficient communication of prioritizations and deliverables, and alterations or loss of information. As a consequence, agile practices need to be extended to fit the identified needs.

Large-scale intra-organizational, yet decentralized software projects that involve various self-contained organizational units require architecture guidelines to coordinate development. Tool support allows for managing architecture-guideline violations to ensure software quality. However, the decentralized development across units results in significant violation-management hurdles that must be considered. In this paper, we present a set of capabilities required to manage guideline violations within two of these large-scale software projects at Siemens. Their main purpose is process support for resolving violations, aiming to reduce the architects' and developers' effort required to handle them. Moreover, we present a prototype that implements the capabilities.

Many systems in the industrial automation domain include information systems. They manage manufacturing processes and control numerous distributed hardware and software components. In current practice, the development and reuse of such systems is costly and time-consuming, due to the variability of systems' topology and processes. Up to now, product line approaches for systematic modeling and management of variability have not been well established for such complex domains.

The idea of software ecosystems encourages organizations to open software projects for external businesses, governing the cross-organizational development by architectural and other measures. Even within a single organization, this paradigm can be of high value for large-scale decentralized software projects that involve various internal, yet self-contained organizational units. However, this intra-organizational decentralization causes architecture challenges that must be understood to reason about suitable architectural measures. We present an in-depth case study on collaboration and architecture challenges in two of these large-scale software projects at Siemens. We performed a total of 46 hours of semi-structured interviews with 17 leading software architects from all involved organizational units. Our major findings are: (1) three collaboration models on a continuum that ranges from high to low coupling, (2) a classification of architecture challenges, together with (3) a qualitative and quantitative exposure of the identified recurring issues along each collaboration model. Our study results provide valuable insights for both industry and academia: Practitioners that find themselves in one of the collaboration models can use empirical evidence on challenges to make informed decisions about counteractive measures. Researchers can focus their attention on challenges faced by practitioners to make software engineering more effective.

Warehouse management systems (WMS) play a critical role in supply chains and large production processes. WMS pose two crucial challenges for variability modeling and management: Firstly, the physical configuration of each warehouse differs significantly. Numerous different electronic devices like controllers, sensors, and motors are used to automate warehouses. Secondly, the processes running in a warehouse demand control and coordination of these various software and hardware components. These processes have different configurations according to customer requirements. This paper reports on our experiences when applying variability modeling techniques in WMS, in order to improve the degree of reuse and shorten the delivery time to customers. Feature modeling is used to extract commonalities and variability in WMS. More than 200 features were identified, and categorized into three hierarchical layers. Furthermore, we linked the feature models to assets, and utilize feature models to support product derivation. Then, the lessons learned from the experiences are discussed. Based on these experiences, we conclude that feature modeling can be applied nicely for scoping features and tracing features to assets, but is not comprehensive enough to support automated configuration during product derivation in WMS.

Large-scale software-intensive systems are often considered as systems of systems (SoS) comprising multiple heterogeneous but interrelated systems. The engineering of SoS often involves the derivation of system variants from multiple interrelated product lines to meet the overall requirements. If multiple teams and experts are involved in the configuration of these individual systems, their individual configuration choices may conflict with each other or violate constraints. This paper illustrates industrial challenges based on a previously conducted case study on distributed configuration in multi product lines. We then present CoDiM, a tool-supported approach for defining and checking constraints in distributed configuration of an SoS. Our approach is integrated in the product line tool suite DOPLER developed in cooperation with industry partners. An application scenario from a real-world multi product line demonstrates how our approach allows detecting violations of constraints during distributed configuration of an SoS. The approach provides immediate feedback to configurers during product derivation and enables the dynamic definition of constraints even during configuration time to accommodate changes. CoDiM further supports constraint templates which can be parameterized to allow their reuse in different multi product line configurations.

Today's large-scale software systems are frequently based on system-of-systems architectures comprising multiple heterogeneous systems. Multi product lines have been presented as an approach to ease their development through systematic reuse. In multi product lines different users and teams are involved in product derivation. Users configure the involved product lines in a collaborative and distributed manner. It is challenging to ensure awareness regarding the configuration choices of users configuring related product lines in such a setting. We present a tool-supported approach that aims at increasing awareness by discovering dependencies and sharing configuration information during the collaborative and distributed product derivation of a multi product line. Our approach uses a bulletin board mechanism for sharing configuration information. It further implements a request-publish-subscribe mechanism for revealing possible dependencies and presenting the information relevant to the users' specific configuration tasks. We evaluated the usability and utility of our approach in a two-phase study involving twelve industrial experts. The evaluation demonstrates the usability and utility of our approach to foster awareness regarding configuration dependencies during the distributed product derivation of a multi product line.

Tool support that checks for configuration errors and generates product parts from configurations can significantly improve on product derivation in product line engineering. Up to now, however, derivation tools commonly disregard the staged derivation process. They do not restrict configuration consistency checks to process entities such as configuration stages, stakeholders, or build tasks. As a result, constraints that are only valid for certain process entities must either be checked permanently, leading to false positive errors, or one must refrain from defining them at all.

Managing and developing a set of software products jointly using a software product line approach has achieved significant productivity and quality gain in the last decade. More and more, product lines now are becoming themselves entities that are sold and bought in the software supply chain. Customers build more specialized product lines on top of them or derive themselves the concrete products. As customers have different requirements, whole product lines now may vary depending on customer needs—they need to be customized. Current approaches going beyond the scope of one product line do not provide appropriate means for customization. They either are tailored to specific implementation techniques, only regard customization on few levels (e.g., only source code level), or imply a lot of manual effort for performing the customization.