Abstract Experimentation has become a cornerstone of agile, evidence-driven software development. To explore how it may evolve in the coming decade, we collected qualitative data from 58 experts across the global experimentation community. Using an inductive thematic analysis inspired by the Gioia methodology, we identified six interrelated trends that capture how practitioners envision the future of experimentation: AI-augmented workflows, segment-level personalization, platformization and warehouse-native architectures, expansion beyond web contexts, rigor at scale, and cultural capability building. These trends highlight experimentation’s evolution from a technical testing practice toward a socio-technical learning system. Building on these insights, the paper outlines a practice-inspired research agenda for the next decade of evidence-based product development.

Abstract Experimentation programs are vital for enabling data-driven decision-making within product development. However, evaluating their overarching success remains a significant challenge. Current metrics, such as conversion rates, primarily focus on individual experiments, leaving a gap in assessing broader program efficiency and impact. This paper addresses this gap by presenting a structured overview and analysis of 18 program-level metrics, categorized into six domains: Volume, Outcome-Based, Quality, Engagement, Process Efficiency and Strategic Alignment. Metrics such as experimentation throughput, time-to-decision and experimentation coverage are examined for their implications on operational efficiency, cultural adoption, and strategic alignment. Based on interviews with 48 experimentation practitioners, this work provides a description of these metrics and discusses their benefits and challenges. The results offer actionable insights for advancing experimentation practices and aligning them with organizational goals.