Abstract Models play a critical role in supporting Verification and Validation activities. However, they are often unavailable in practice, such as for legacy systems or third-party components. Model learning addresses this gap through two main approaches: passive learning, which infers models from existing execution traces, and active learning, which interacts with the system under learning (SUL) to generate more general models, but at higher computational cost. In this work, we propose a novel application of Transformer architectures to implicitly learn generic system models from execution traces alone, combining the efficiency of passive learning with the generality of active learning. We design and evaluate two Transformer-based architectures: one focused on exploitation, achieving over $$\varvec{95\%}$$ valid trace generation; and another focused on exploration, generating approximately $$\varvec{60\%}$$ novel, previously unseen traces. Our methods outperform state-of-the-art model learning techniques, demonstrating that Transformers can achieve results comparable to active learning while requiring significantly fewer resources.

Failed Error Propagation greatly reduces the effectiveness of Software Testing by masking faults present in the code. This situation happens when the System Under Test executes a faulty statement, the state of the system is affected by this fault, but the expected output is observed. Therefore, it is a must to assess its impact in the testing process. Squeeziness has been shown to be a useful measure to assess the likelihood of fault masking in deterministic systems. The main goal of this paper is to define a new Squeeziness notion that can be used in a scenario where we may have non-deterministic behaviours. The new notion should be a conservative extension of the previous one. In addition, it would be necessary to evaluate whether the new notion appropriately estimates the likelihood that a component of a system introduces Failed Error Propagation. We defined our black-box scenario where non-deterministic behaviours might appear. Next, we presented a new Squeeziness notion that can be used in this scenario. Finally, we carried out different experiments to evaluate the usefulness of our proposal as an appropriate estimation of the likelihood of Failed Error Propagation. We found a high correlation between our new Squeeziness notion and the likelihood of Failed Error Propagation in non-deterministic systems. We also found that the extra computation time with respect to the deterministic version of Squeeziness was negligible. Our new Squeeziness notion is a good measure to estimate the likelihood of Failed Error Propagation being introduced by a component of a system (potentially) showing non-deterministic behaviours. Since it is a conservative extension of the original notion and the extra computation time needed to compute it, with respect to the time needed to compute the former notion, is very small, we conclude that the new notion can be safely used to assess the likelihood of fault masking in deterministic systems.

Chess engines are computer programs that analyse chess positions. The goal of this analysis is to decide which player has an advantage and evaluate how big the advantage is. Using this analysis, chess engines are really powerful players who can consistently beat the best (human) players. Even though these programs are fantastic players, we cannot be sure that the code is fault free because it is very difficult to test them. In particular, we face the oracle problem: if the chess engine plays better than any potential tester, how can a tester claim that a certain evaluation is wrong or that a suggested move is not the best one? The main goal of our work is to provide a metamorphic testing tool to evaluate chess engines. In particular, we are interested in looking for inconsistent behaviours in the best publicly available chess engine, Stockfish, but we would also like to consider other chess engines. We developed a metamorphic testing solution to validate chess engines. First, we defined metamorphic relations that might reveal inconsistent behaviours. The underlying idea was that the evaluation of related positions should be the same. For example, if we consider a position and rotate all the pieces with respect to the central axis, then both positions should have the same evaluation. One of our main priorities was to have a fully automatised tool. Source inputs are obtained from available datasets while follow-up inputs are automatically computed by applying sound transformations to the source inputs with respect to the corresponding metamorphic rule. In order to assess the usefulness of our work, we applied it to analyse a dataset with more than 40,000 positions. Empirical evidence validates the usefulness of our work to analyse the best available chess engine, Stockfish. Our tool revealed non-negligible deviations from the expected behaviour in Stockfish for all the MRs. Additional experiments showed that our tool can be easily used to analyse other chess engines such as Komodo, Houdini and Gull. The experiments demonstrate the usefulness of our approach to identify issues in the latest version of the widely recognised to be the best chess engine: Stockfish (version 15, released in April 2022). Our tool is flexible and can be easily extended with metamorphic relations that can be defined in the future by either us or other users. Since all our metamorphic relations are implemented and the code is freely available, users can use them as a pattern to implement new relations.

In personalised health, small datasets with missing data are quite common. Current Machine Learning methods are unable to process such datasets in a meaningful way due to the huge data volume requirement. To address this problem, we propose a new Small and iNcomplete Dataset Analyser (SaNDA) to process such datasets in a meaningful way. Due to the characteristics of these datasets and the criticality of the domain, an explainable method is mandatory to facilitate decision-making interpretation. Thus, SaNDA prioritises explainability over efficiency by design. We evaluated our proposal against Random Forest as a baseline for explainable methods, and against gcForest as state-of-the-art for small datasets. We observed that our proposal outperforms Random Forest when there is more missing data and/or lower number of entries in the dataset, obtaining less favourable results over larger, well-curated datasets. It is also preferable than gcForest due to its explainability and privacy protection capabilities. Given the difficulties in obtaining complete, reliable data in the healthcare field, we consider that our proposal could be useful for practitioners.

Software testing is fundamental to ensure the reliability of software. To properly test software, it is critical to generate test suites with high fault finding ability. We propose a new method to generate such test suites: a coverage-based grammar-guide genetic programming algorithm. This evolutionary computation based method allows us to generate test suites that conform with respect to a specification of the system under test using the coverage of such test suites as a guide. We considered scenarios for both black-box testing and white-box testing, depending on the different criteria we work with at each situation. Our experiments show that our proposed method outperforms other baseline methods, both in performance and execution time.

Software Testing usually considers programs with parameters ranging over simple types. However, there are many programs using structured types. The main problem to test these programs is that it is not easy to select a relatively small test suite that can find most of the faults in these programs. In this paper we present a framework to generate test suites for unit testing of methods which have trees as parameters. We combine classical mutation testing with Evolutionary Computation techniques to evolve a population of trees. The final goal is to obtain a set of trees, representing, good test cases, that will be used as the test suite to test the corresponding method.

Software product lines are an excellent mechanism in the development of software. Testing software product lines is an intensive process where selecting the right features where to focus it can be a critical task. Selecting the best combination of features from a software product line is a complex problem addressed in the literature. In this paper, we address the problem of finding the combination of features with the highest probability of being requested from a software product line with probabilities. We use Evolutive Computation techniques to address this problem. Specifically, we use the Ant Colony Optimization algorithm to find the best combination of features. Our results report that our framework overcomes the limitations of the brute force algorithm.

Mutation Testing is an effective testing technique that relies in the generation of mutants from the system under test. The main limitation of this technique is that the potential number of mutants is usually huge. Therefore, it is important to classify and select mutants in order to avoid repetitive, useless or excessive computations, and biased results. In this paper we focus on avoiding too many executions and/or biased results by classifying mutants into two categories: hard-to-kill and easy-to-kill mutants. We propose a new swarm intelligence algorithm to classify a set of mutants between those two classes and we show how our algorithm compares to other approaches.

Particle Swarm Optimization (PSO) has been successfully applied to find good solutions through a guided search. This optimization technique usually works with vectors as individuals of the population conforming the search space. Nevertheless, there exist problems such that the search space cannot be transformed into a vector search space. In this paper we propose a novel technique based on the intuition behind PSO but overcoming its limitations concerning search spaces. Specifically, we present a PSO framework where the individuals conforming the search space are tree-like structures. In particular, our framework naturally includes classical PSO but also search spaces where elements are structures that can be represented as trees (in addition to usual trees, linear structures such as lists, queues and stacks).