This paper presents an automated, model based test validation and oracle approach for systems with complex input and output structures, such as Data Acquisition (DAQ) systems, which are common in many sectors including the satellite communications industry. We present a customised modelling methodology for such systems and a tool that automatically validates test inputs and, after test execution, applies an oracle that is based on mappings between the input and output. We also apply our proposed approach and tool to a complex industrial DAQ system and investigate the scalability and effectiveness of the approach in validating test cases, the DAQ system, or its specifications (captured as models). The results of the case study show that the approach is indeed scalable with respect to two dimensions: (1) model size and (2) test validation and oracle execution time. The size of the model for the DAQ system under study remains within practical bounds, and far below that of typical system models, as it includes a class diagram with 68 classes and 49 constraints. The developed test validation and oracles tool can handle satellite transmission files up to two GB within practical time constraints, taking, on a standard PC, less than three minutes for test validation and less than 50 minutes for applying the oracle. The approach was also effective in automatically applying the oracle successfully for the actual test suite of the DAQ system, accurately identifying all issues and violations that were expected, thus showing that an approach based on models can be sufficiently accurate.

Model-Based Testing (MBT) is a widely used methodology for generating tests aiming to ensure that the system behaviour conforms to its specification. Recently, it has been successfully applied for testing certain security properties. However, for the success of this approach, it is an important prerequisite to consider the correctness of test models with respect to the given security property. In this paper we present an approach for smart-card specific security properties that permits to validate the system with MBT from test schemas. We combine this MBT approach with UMLsec security verification technique, by using UMLsec stereotypes to verify the model w.r.t. given security properties and gain more confidence in the model. We then define an automatic procedure to generate security test from the UMLsec model via so-called "test schemas". We validate this approach on a fragment of the Global Platform specification and report on available tool support.