Validating a configurable software system is challenging, as there are potentially millions of configurations, which makes testing each configuration individually infeasible. Thus, existing sampling algorithms allow to compute a representative subset of configurations, called sample, that can be tested instead. However, sampling on the set of configurations may miss potential error sources on implementation level. In this paper, we present solution-space sampling, a concept that mitigates this problem by allowing to sample directly on the implementation level. We apply solution-space sampling to six real-word, automotive product lines and show that it produces up to 56 % smaller samples, while also covering all potential error sources missed by problem-space sampling.

A software product line allows to derive individual software products based on a configuration. As the number of configurations is an indicator for the general complexity of a software product line, automatic #SAT analyses have been proposed to provide this information. However, the number of configurations does not need to match the number of derivable products. Due to this mismatch, using the number of configurations to reason about the software complexity (i.e., the number of derivable products) of a software product line can lead to wrong assumptions during implementation and testing. How to compute the actual number of derivable products, however, is unknown. In this paper, we mitigate this problem and present a concept to derive a solution-space feature model which allows to reuse existing #SAT analyses for computing the number of derivable products of a software product line. We apply our concept to a total of 119 subsystems of three industrial software product lines. The results show that the derivation scales for real world software product lines and confirm the mismatch between the number of configurations and the number of products.