Platform ecosystems have revolutionized value creation across numerous industries, inducing technical leaps and scaling artifact generation in scales and speeds unattainable through traditional vertically integrated or single-firm models. These ecosystems rely on collaborative interactions among actors to co-create and reuse value. Such ecosystems are socio-technical environments which require complex governance and orchestration strategies to ensure ecosystem health and performance from financial, technical, and social perspectives. Consequently, monitoring the performance of platform ecosystems requires non-primitive metrics as factors contributing to ecosystem performance are multifaceted compared to conventional software settings. Effective orchestration of platform ecosystems requires relies on access to real-time quantitative performance indicators. The existing literature offers various quantitative health metrics and performance indicators for platform ecosystems, yet these are dispersed across multiple studies and often embedded in abstract models or found within generic analytics systems. This research reviews existing quantitative real-time health metrics and performance indicators of platform ecosystems. We identified 417 distinct metrics after eliminating duplicates and incomplete definitions, and refined 168 of these metrics to be calculable in real-time using a consistent framework for definition, nomenclature, and quantification. Furthermore, we compiled existing real-time ecosystem health monitoring methods into a reference architecture and tested its feasibility in two active platform ecosystems. The study yields four key contributions: a practical catalog of platform ecosystem health metrics; a reference architecture for creating real-time ecosystem health monitoring solutions, demonstrated through implementation in two operational platform ecosystems; industry-relevant insights for practitioners; and a discussion of potential future research directions.

Context: Software ecosystems have transformed many industries, redefining collaboration and value co-creation. The success of such ecosystems depends on the dynamism of the network of users on its different sides. Consequently, decision-making in such multifaceted and interconnected environments is more complex than in conventional software products. On-line controlled experiments are considered the gold standard for aiding decision-making in software engineering processes. Experiments are extensively used to reduce bias and estimation noise for design, engineering, and business decisions. However, experimentation in software ecosystems is inherently more com-plex as it deals with atypical sources of bias and technical complications. Primary studies of experimentation approaches in software ecosystems are scattered across multiple domains and disciplines, and secondary research on the topic is scarce as highlighted in different tertiary studies. Hence, we conducted this study. Objectives: To explore primary research on experimentation in software ecosystems; Summarize current approaches, toolboxes, and solutions that practitioners and researchers, facing similar problems, can use to inform their approaches; To outline underexplored research areas and provide recommendations for practitioners. Method: We conducted a systematic literature review. The search strategy, application of exclusion and inclusion criteria, and subsequent quality assessment resulted in 63 relevant studies. Data extraction process was designed and carried out to collect data relevant to the study objectives. The extracted data under-went descriptive and thematic syntheses and analyses, in addition to cross-analysis on relevant axes. Contributions: The study resulted in four contributions. First, a distillation of the themes and patterns in the available research on the topic. Second, a practical summary of the experimental designs specific to each software ecosystem type. Third, an actionable road map for practitioners in order to achieve exper-imentation maturity in software ecosystems. Fourth, an outline of the underexplored research areas.

Industrial software ecosystems refer to a network of interdependent actors, co-creating value through a shared technological platform specifically tailored to industrial sectors. Developing, maintaining, and orchestrating such platforms involves many challenges that require complex decision making. Experimentation can help alleviate this complexity and reduce decision uncertainty and bias. However, experimentation requires certain organizational, infrastructural, and data-related prerequisites which can be uniquely challenging to achieve in industrial software ecosystems. Through semi-structured interviews with 25 industry professionals involved in various roles across 17 ecosystems, we analyze the difficulties faced in conducting effective experiments in such environments. The interview protocol covered aspects related to the methodologies, data handling processes, and current experimentation practices, as well as the challenges faced by practitioners who engage in experimentation initiatives. The study findings reveal technical, organizational, and market-related challenges, detailing the complexities facing experimentation initiatives in industrial software ecosystems. The findings are presented in an actionable manner, following a model that allows business-oriented alignment of architecture, process, and organizational evolution strategies. The study identifies key impediments, such as data integration difficulties, stringent regulatory environments, and prevailing organizational cultures that hinder continuous experimentation practices. Our analysis provides a foundation for understanding the unique challenges facing experimentation efforts in industrial software ecosystems and offers insights into potential strategies to improve the effectiveness of these initiatives.

The emergence of platform ecosystems has transformed the business landscape in many industries, giving rise to novel modes of interorganizational cooperation and value co-creation, as well as unconventional challenges. The vast traces of data generated by platform ecosystems makes them ripe for the use of analytics and data-driven methods aimed at improving their health, performance, business outcomes, and evolution. However, the research on the application of analytics within platform ecosystems is limited and spread across multiple disciplines. To address this gap, we conducted a systematic literature review on the application of analytics and data-driven methods and practices within platform ecosystems. A total of 56 studies were reviewed, and underwent data extraction, analysis, and synthesis processes. In addition to presenting themes and patterns in the recent and relevant literature on platform ecosystems analytics, our review offers the following outcomes: an actionable overview of the analytics toolbox currently used within platform ecosystems—spanning domains such as machine learning, deep learning, data science, modelling, simulation, among others—; a roadmap for practitioners to achieve analytics maturity; and a summary of underexplored research areas.