Background: One way to reduce test cost is to automatise testing tasks. Model-based testing (MBT) tools use system behaviour models as inputs to automatically generate tests. In the literature, few experiments evaluate the impact of using an MBT tool in test case productivity, test coverage, and bug detection rate.Aims: This work consists of the first part of the evaluation of the impacts of using an MBT tool whose input model is based on natural language, the TaRGeT. This MBT tool uses as inputs use case models authored using natural language. We assess the effects of using an MBT tool in test case productivity (number of test steps produced per hour).Method: A quasi-experiment was carried out for evaluating test case productivity when creating functional tests manually vs the productivity when designing tests using TaRGeT. The application size and subjects experience were controlled, and the collected data statistically analysed.Results: The results show the mean productivity when using TaRGeT is 30% higher when compared to the mean productivity when designing tests manually. Despite this difference, it was not possible to detect statistical differences in productivity when using TaRGeT or not.Conclusions: We discuss possible reasons for this behaviour and other findings, moreover we present lessons learned for future experiments.

We propose a semantics for Sequence Diagrams based on the COMPASS Modelling Language (CML): a formal specification language to model systems of systems. A distinguishing feature of our semantics is that it is defined as part of a larger effort to define the semantics of several diagrams of SysML, a UML profile for systems engineering. We have defined a fairly comprehensive semantics for Sequence Diagrams, which comprises sequential and parallel constructors, loops, breaks, alternatives, synchronous and asynchronous messages. We illustrate our semantics with a scenario of a case study of a system of systems. We also discuss an analysis strategy which involves an integrated view of several diagrams.

The order in which tests are executed can significantly impact the total test execution time. In this paper, we evaluate two test prioritization techniques (manual and automatic) in the context of mobile phone testing. The manual technique produces test sequences created by test experts, while the automatic one generates sequences mechanically based on the permutation of the tests. Both techniques take into account a data reuse: the more the data is reused among tests, the faster the sequence is executed. In order to evaluate the benefits of these two techniques, we carried out an experiment with 8 testers and 2 test suites arranged in a 2times2 Latin square design replicated four times. The automatic technique reduced approximately 25% of the data generation time and 13.5% of the execution time. The automatic technique is clearly better than the manual one with respect to the generation of sequences. Our experiment showed that the automatic technique also generates sequences whose execution is faster than those created manually by test experts.