Combinatorial Test Design (CTD) is an effective test planning technique that reveals faults resulting from feature interactions in a system. The standard application of CTD requires manual modeling of the test space, including a precise definition of restrictions between the test space parameters, and produces a test suite that corresponds to new test cases to be implemented from scratch. In this work, we propose to use Interaction-based Test-Suite Minimization (ITSM) as a complementary approach to standard CTD. ITSM reduces a given test suite without impacting its coverage of feature interactions. ITSM requires much less modeling effort, and does not require a definition of restrictions. It is appealing where there has been a significant investment in an existing test suite, where creating new tests is expensive, and where restrictions are very complex. We discuss the tradeoffs between standard CTD and ITSM, and suggest an efficient algorithm for solving the latter. We also discuss the challenges and additional requirements that arise when applying ITSM to real-life test suites. We introduce solutions to these challenges and demonstrate them through two real-life case studies.

Combinatorial test design (CTD) is an effective test planning technique that reveals faults that result from feature interactions in a system. The test space is manually modeled by a set of parameters, their respective values, and restrictions on the value combinations. A subset of the test space is then automatically constructed so that it covers all valid value combinations of every t parameters, where t is usually a user input. In many real-life testing problems, the relationships between the different test parameters are complex. Thus, precisely capturing them by restrictions in the CTD model might be a very challenging and time consuming task. From our experience, this is one of the main obstacles in applying CTD to a wide range of testing problems. In this paper, we introduce two new constructs to the CTD model, counters and value properties, that considerably reduce the complexity of the modeling task, allowing one to easily model testing problems that were practically impossible to model before. We demonstrate the impact of these constructs on two real-life case studies.

Combinatorial test design (CTD) is an effective test planning technique that systematically exercises interactions between parameters of the test space. The test space is manually modeled by a set of parameters, their respective values, and restrictions on the value combinations. A subset of the test space is then automatically constructed so that it covers all valid value combinations of every t parameters, where t is a user input. This paper describes patterns that we have found to be recurring in combinatorial models, i.e., recurring properties of the modeled test spaces. These patterns are often hard to identify and capture correctly in a model, thus are common pitfalls in combinatorial modeling. We describe these patterns, supply methods for identifying them, and suggest simple yet effective solutions for them.

Large systems generate immense quantities of code coverage data. A user faced with the task of analyzing this data, for example, to decide on test areas to improve, faces a 'needle in a haystack' problem.

Code coverage is a common aid in the testing process. It is generally used for marking the source code segments that were executed and, more importantly, those that were not executed.

Reliability Accessibility and Serviceability (RAS) software deals with hardware-related processes that typically include manual operations such as replacing components. The necessity to perform manual operations inhibits automated tests, reduces the scope of unit testing, and makes it challenging to create a regression test suite for RAS.