Software is typically configurable by means of compile-time or runtime variability. As testing every valid configuration is infeasible, T-Wise sampling has been proposed to systematically derive a relevant subset of the configurations for testing to cover interactions among t features. Practitioners started to apply T-Wise sampling algorithms, but can often only test samples partially due to restricted resources and compare those partial samples based on their T-Wise coverage. However, there is no consensus in the literature on how to compute the T-Wise coverage in the literature. We propose the first systematic framework to define coverage metrics for T-Wise feature interactions. These metrics differ in the features and feature interactions being considered. We found evidence for at least six different metrics in the literature. In an empirical evaluation, we show that for a partial sample the coverage differs up to 21 % and for some metrics only half of the feature interactions need to be covered. As a long-term impact, our work may help to improve the efficiency and effectiveness of both, T-Wise sampling and coverage computations.

Configurable software systems, such as software product lines, enable the generation of products based on configurations tailored to specific requirements by combining reusable features. A key challenge in product lines lies in combinatorial interaction testing, which ensures that all possible feature combinations are tested to identify configurations that may fail. When a configuration fails, pinpointing the feature or the feature interaction causing the fault is crucial. However, fault masking - where faulty interactions remain undetected because other features or interactions could override their effects - potentially hinders the effective identification of faults in product lines. Despite the potential of missing critical interaction faults, fault masking in product lines has received limited attention in existing research. To address this gap, we investigate and analyze on already identified faults of the real-world product line JHipster and quantitatively analyze these faults in terms of masking. In our case study, we find evidence of the existence of fault masking in JHipster and how the detectability of masked faults is influenced. For one feature-interaction fault in JHipster, we miss to identify 17.6% of all configurations containing this fault due to masking effects. By analyzing masked faults of a real-world product line, we raise awareness of investigating feature-interaction masking further in software product lines.