ABSTRACT Clustering is increasingly being used to select the appropriate test suites. In this paper, we apply this approach to regression testing. Regression testing is the practice of verifying the robustness and reliability of software by retesting after changes have been made. Creating and maintaining functional regression tests is a laborious and costly activity. To be effective, these tests must represent the actual user journeys of the application. In addition, an optimal number of test cases is critical for the rapid execution of the regression test suite to stay within the time and computational resource budget as it is re‐run at each major iteration of the software development. Therefore, the selection and maintenance of functional regression tests based on the analysis of application logs has gained popularity in recent years. This paper presents a novel approach to improve regression testing by automating the creation of test suites using user traces fed into clustering pipelines. Our methodology introduces a new metric based on pattern mining to quantify the statistical coverage of prevalent user paths. This metric helps to determine the optimal number of clusters within a clustering pipeline, thus addressing the challenge of suboptimal test suite sizes. Additionally, we introduce two criteria, to systematically evaluate and rank clustering pipelines. Experimentation involving 33 variations of clustering pipelines across four datasets demonstrates the potential effectiveness of our automated approach compared with manually crafted test suites. (All the experiments and data on Scanner, Spree and Booked Scheduler are available at https://github.com/frederictamagnan/STVR2024 .) Then, we analyse the semantics of the clusters based on their principal composing patterns.

In maritime traffic surveillance, detecting illegal activities, such as illegal fishing or transshipment of illicit products is a crucial task of the coastal administration. In the open sea, one has to rely on Automatic Identification System (AIS) message transmitted by on-board transponders, which are captured by surveillance satellites. However, insincere vessels often intentionally shut down their AIS transponders to hide illegal activities. In the open sea, it is very challenging to differentiate intentional AIS shutdowns from missing reception due to protocol limitations, bad weather conditions or restricting satellite positions. This paper presents a novel approach for the detection of abnormal AIS missing reception based on self-supervised deep learning techniques and transformer models. Using historical data, the trained model predicts if a message should be received in the upcoming minute or not. Afterwards, the model reports on detected anomalies by comparing the prediction with what actually happens. Our method can process AIS messages in real-time, in particular, more than 500 Millions AIS messages per month, corresponding to the trajectories of more than 60 000 ships. The method is evaluated on 1-year of real-world data coming from four Norwegian surveillance satellites. Using related research results, we validated our method by rediscovering already detected intentional AIS shutdowns.

Maritime surveillance is essential to avoid illegal activities and for environmental protection. However, the unlabeled, noisy, irregular time-series data and the large area to be covered make it challenging to detect illegal activities. Existing solutions focus only on trajectory reconstruction and probabilistic models that do ignore the context, such as the neighboring vessels. We propose a novel representation learning method that considers both temporal and spatial contexts learned in a self-supervised manner, using a selection of pretext tasks that do not require to be labeled manually. The underlying model encodes the representation of maritime vessel data compactly and effectively. This generic encoder can then be used as input for more complex tasks lacking labeled data.

Manual test suites are typically described by natural language, and over time large manual test suites become disordered and harder to use and maintain. This paper focuses on the challenge of providing tool support for refactoring such test suites to make them more usable and maintainable. We describe how we have applied various machine-learning and NLP techniques and other algorithms to the refactoring of manual test suites, plus the tool support we have built to embody these techniques and to allow test suites to be explored and visualised. We evaluate our approach on several industry test suites, and report on the time savings that were obtained.

We present an approach to use techniques of model-based testing (MBT) applied on security cryptographic components. This application of MBT is done in the context of a qualification testing phase made by an entity independent from designers, developers and sponsors of the cryptographic components under test. This qualification phase targets both hardware and software cryptographic components and the testing activities cover functional and security testing objectives. In this context, we present the application of MBT for two cryptographic components (one hardware and one software) and show the complementary of test selection criteria based on one side on a structural coverage of the behavioral model used for test generation, and on the other side on a test purpose approach to meet some security test objectives. The test purpose language used in this project is novel and has been designed to complete behavioral model coverage criteria in the MBT process.

This paper presents Smartesting CertifyIt, a general-purpose model-based testing solution dedicated to the IT domain. In the process supported by this model-based testing solution, test generation models are developed using a subset of UML and BPMN notations. The automated test generation phase is led by several kinds of test selection criteria such as requirements coverage or scenario-based criteria. It targets both manually executed and automated test cases generation, and supports the publication of generated tests in a test management tool. In this paper, we describe the model-based testing process with Smartesting Certify-It for Enterprise IT applications based on composing business process and behavioral models.

SUMMARY Model‐based testing (MBT) relies on models of a system under test and/or its environment to derive test cases for the system. This paper discusses the process of MBT and defines a taxonomy that covers the key aspects of MBT approaches. It is intended to help with understanding the characteristics, similarities and differences of those approaches, and with classifying the approach used in a particular MBT tool. To illustrate the taxonomy, a description of how three different examples of MBT tools fit into the taxonomy is provided. Copyright © 2011 John Wiley & Sons, Ltd.

This paper proposes a new family of model-based coverage criteria, based on formalizing boundary-value testing heuristics. The new criteria form a hierarchy of data-oriented coverage criteria, and can be applied to any formal notation that uses variables and values. They can be used either to measure the coverage of an existing test set, or to generate tests from a formal model. We give algorithms that can be used to generate tests that satisfy the criteria. These algorithms and criteria have been incorporated into the BZ-TESTING-TOOLS (BZ-TT) tool-set for automated test case generation from B, Z and UML/OCL specifications, and have been used and validated on several industrial applications in the domain of critical software, particularly smart cards and transport systems.

Abstract This paper presents the results of a case study on generating test cases for a fragment of the smart card GSM 11‐11 standard. The generation method is based on an original approach using the B notation and techniques of constraint logic programming with sets. The GSM 11‐11 technical specifications were formalized with the B notation. From this B specification, a system of constraints was derived, equivalent to this formal model. Using a set constraint solver, boundary states were computed and test cases were obtained by traversing the constrained reachability graph of the specifications. The purpose of this project was to evaluate the contribution of this testing environment, called B‐TESTING‐TOOLS, in an industrial process on a real life‐size application, by comparing the generated test sequences with the already used and high‐quality manually‐designed tests. This comparison enabled us to validate our approach and showed its effectiveness in the validation process of critical applications: the case study gives a wide coverage (about 85%) of the generated tests compared to the pre‐existing tests and a saving of 30% in test design time. Copyright © 2004 John Wiley & Sons, Ltd.

Abstract BZ‐Testing‐Tools (BZ‐TT) is a tool‐set for automated model‐based test case generation from B abstract machines and Z specifications. BZ‐TT uses boundary testing as well as cause–effect testing on the basis of the formal model. It has been used and validated on several industrial case studies in the domain of critical software: in particular for smart card applications and automotive embedded systems. The main idea of BZ‐TT is to compute a boundary goal for each effect of the operations of the model and then to compute a preamble sequence of operations to place the system under test in such a state that satisfies the goal. In this paper, the preamble computation search strategies used in BZ‐TT are presented. More precisely, two algorithms based respectively on forward chaining and backward chaining are compared. These algorithms both use a customized set constraint solver, which is able to animate the formal model. These algorithms differ, however, in their capacity to reach the boundary goals efficiently. The results of applying the tools to an industrial windscreen wiper controller application are presented. Copyright © 2004 John Wiley & Sons, Ltd.

Abstract BZ‐TESTING‐TOOLS (BZ‐TT) is a tool set for automated test case generation from B and Z specifications. BZ‐TT uses boundary and cause–effect testing on the basis of the formal model. It has been used and validated on several industrial applications in the domain of critical software, particularly smart card and transport systems. This paper presents the test coverage criteria supported by BZ‐TT. On the one hand, these correspond to various classical structural coverage criteria, but specialized to the case of B abstract machines. The paper gives algorithms for these in Prolog. On the other hand, BZ‐TT introduces new coverage criteria for complex data structures, based on boundary analysis: this paper defines weak and strong state‐boundary coverage, input‐boundary coverage and output‐boundary coverage. Finally, the paper describes how BZ‐TT presents a unified view of these criteria to the validation engineer, and allows him or her to control the test case explosion on a coarse basis (choosing from a range of coverage criteria) as well as a fine basis (selecting options for each state or input variable). Copyright © 2004 John Wiley & Sons, Ltd.

The paper presents an original method to generate test sequences. From formal specifications of the system to be tested, an equivalent system of constraints is derived, and then the domain of each state variable of this system is partitioned into subdomains. Using this partition, limit states are computed with a specific solver that uses constraint logic programming with sets. This specific solver is then used to build test sequences by traversing the constrained reachability graph of the specifications. Finally, the formal specifications are used as an oracle by using them to determine the expected output for a given input. The results of an industrial case-study of the Smart Card GSM 11-11 standard are presented and discussed.

This paper deals with consistency techniques over sequences constraints embedded in Constraint Logic Programming CLPS. CLP Sequences constraints are defined over Hereditarily Homogeneous Finite Sets HHFS built on atomic elements to characterise a family of admissible sequences. The relations we are dealing with are classical sets relations (/spl isin/,/spl sub/,=,/spl ne/) and sequencing relations as potential, metric and range constraints. We define the semantics of these relations with a characteristic range function. The consistency techniques used are incremental reduction of the normal form based on a tree like representation called P-Q-R trees. This allows us to reduce the set of admissible sequences before generating solutions.>

Scheduling the movements of a hoist in printed circuit board electroplating is known is the hoist scheduling problem (HSP). The objective of the HSP is to find an optimal cyclic sequence of moves for hoists. Results are already available for this problem, using classical operational research tools. They are briefly presented. Most of the proposed solutions are limited to the very specific initial model and are consequently difficult to apply in industrial problems. The constraint logic programming (CLP) approach to the problem is presented. A CLP model of the problem that allows modeling and solving more realistic cases is presented, and the results and the analysis of the solutions are outlined.>