As automated systems advance in complexity, comprehensive testing becomes crucial, particularly for human–robot interaction (HRI) environments where human unpredictability creates ill-defined testing domains that challenge conventional software testing approaches. In interactive robotics, evaluation criteria extend beyond performance to include critical safety considerations. This paper introduces a novel automated testing framework combining runtime monitoring with constraint-based techniques for HRI environments. The framework employs a three-level cognitive oracle architecture – observation, interpretation, and diagnosis – that automatically evaluates the correctness of human and robot actions without requiring expert human intervention. The approach uses constraint-based modeling to handle the non-deterministic nature of HRI scenarios while ensuring safety compliance. Validation through five test cases in a refrigerator disassembly simulation demonstrates the framework’s effectiveness in detecting safety violations and procedural errors under environmental uncertainties.

In many domains, engineers build simulation models (e.g., Simulink) before developing code to simulate the behavior of complex systems (e.g., Cyber-Physical Systems). Those models are commonly heavy to simulate which makes it difficult to execute the entire test suite. Furthermore, it is often difficult to measure white-box coverage of test cases when employing such models. In addition, the historical data related to failures might not be available. This paper proposes a cost-effective approach for test case selection that relies on black-box data related to inputs and outputs of the system. The approach defines in total five effectiveness measures and one cost measure followed by deriving in total 15 objective combinations and integrating them within Non-Dominated Sorting Genetic Algorithm-II (NSGA-II). We empirically evaluated our approach with all these 15 combinations using four case studies by employing mutation testing to assess the fault revealing capability. The results demonstrated that our approach managed to improve Random Search by 26% on average in terms of the Hypervolume quality indicator.

The test case generation and prioritization of industrial cyber-physical systems face critical challenges, and simulation-based testing is one of the most commonly used techniques for testing these complex systems. However, simulation models of industrial CPSs are usually very complex, and executing the simulations becomes computationally expensive, which often make it infeasible to execute all the test cases. To address these challenges, this paper proposes a multi-objective test generation and prioritization approach for testing industrial CPSs by defining a fitness function with four objectives and designing different crossover and mutation operators. We empirically evaluated our fitness function and designed operators along with five multi-objective search algorithms [e.g., nondominated sorting genetic algorithm (NSGA-II)] using four case studies. The evaluation results demonstrated that NSGA-II achieved significantly better performance than the other algorithms and managed to improve random search for on average 43.80% for each objective and 49.25% for the quality indicator hypervolume.

The test case generation of Cyber-Physical Systems (CPSs) face critical challenges that traditional methods such as Model-Based Testing cannot deal with. As a result, simulation-based testing is one of the most commonly used techniques for testing CPSs despite sometimes being computationally too expensive. This paper proposes a search-based approach which is implemented on top of Non-dominated Sorting Genetic Algorithm II (NSGA-II), the most commonly applied multi-objective search algorithm for cost-effectively generating executable test cases in order to test CPSs. With the aim of guiding the generation of the optimal set of so-called reactive test cases, the approach formally defines three cost-effectiveness measures: requirements coverage, test case similarity and test execution time. Furthermore, we design one crossover operator and three mutation operators (i.e., mutation at test suite level named Mu TS, mutation at test case level named Mu TC and mutation at both levels named Mu BO) for test case generation. We evaluate our approach by comparing with Random Search (RS) using four case studies (one of them is an industrial system). Moreover, we evaluate the three mutation operators using the four case studies. The results of the experiment (with a rigorous statistical analysis) indicated that our approach in conjunction with the crossover operator operation and three mutation operators significantly outperformed RS. In general, Mu BO achieved the best performance among the three mutation operators and managed to improve on average the test execution time by 14%, the requirements coverage by 34%, and the test similarity by 75% as compared with RS.

With buildings accounting for nearly 40 percent of global energy consumption, the improvement of energy efficiency in buildings and districts is a clear opportunity in the fight against climate change. This has led engineering practitioners as well as researchers to propose solutions for smart energy control in buildings. Simulation-based methods permits the early validation of engineering solutions for these smart energy control systems. However, since these solutions involve different engineering disciplines (e.g., software engineering, electrical engineering, etc.), different simulation tools might be employed. We propose a tool that interconnects EnergyPlus, one of the leading open-source tools for building energy simulation, with Crescendo, a tool for designing and modelling cyber-physical systems using formal methods. A preliminary evaluation suggests that the proposed solution enables the simulation between these two tools in an efficient manner.

The paper presents the action research conducted in the context of System Engineering teaching in Embedded Systems Master to improve the motivation of students. These are the steps carried out: Identification of the problem domain, diagnosis of the problem, action hypothesis definition, design of the action plan, the action and its analysis, reflection and conclusions.

Cyber-Physical Systems (CPSs) can be found in many sectors (e.g., automotive and aerospace). These systems are usually configurable to give solutions based on different needs. The variability of these systems is large, which implies they can be set into millions of configurations. As a result, different testing processes are needed to efficiently test these systems: the appropriate configurations must be selected and relevant test cases for each configuration must be chosen as well as prioritized. Prioritizing the order in which the test cases are executed reduces the time for detecting faults in these kinds of systems. However, the test suite size is often large and exploring all the possible test case orders is infeasible. Search algorithms can help find optimal solutions from a large solution space. This paper presents an approach based on weight-based search algorithms for prioritizing the test cases for configurable CPSs. We empirically evaluate the performance of the following algorithms with two case studies: Weight-Based Genetic Algorithms, Random Weighted Genetic Algorithms, Greedy, Alternating Variable Method and Random Search (RS). Our results suggest that all the search algorithms outperform RS, which is taken as a baseline. Local search algorithms have shown better performance than global search algorithms.

Cyber-Physical Systems (CPSs) are often tested at different test levels following "X-in-the-Loop" configurations: Model-, Software- and Hardware-in-the-loop (MiL, SiL and HiL). While MiL and SiL test levels aim at testing functional requirements at the system level, the HiL test level tests functional as well as non-functional requirements by performing a real-time simulation. As testing CPS product line configurations is costly due to the fact that there are many variants to test, test cases are long, the physical layer has to be simulated and co-simulation is often necessary. It is therefore extremely important to select the appropriate test cases that cover the objectives of each level in an allowable amount of time. We propose an efficient test case selection approach adapted to the "X-in-the-Loop" test levels. Search algorithms are employed to reduce the amount of time required to test configurations of CPS product lines while achieving the test objectives of each level. We empirically evaluate three commonly-used search algorithms, i.e., Genetic Algorithm (GA), Alternating Variable Method (AVM) and Greedy (Random Search (RS) is used as a baseline) by employing two case studies with the aim of integrating the best algorithm into our approach. Results suggest that as compared with RS, our approach can reduce the costs of testing CPS product line configurations by approximately 80% while improving the overall test quality.

The use of Model Driven Development (MDD) is increasing in industry. When a Non Functional Requirement (NFR) not considered in the development must be added metamodels, models and also transformations are affected. Tasks for defining and maintaining model transformation rules can be complex in MDD. Model Transformation By Example (MTBE) approaches have been proposed to ease the development of transformation rules. In this paper an approach based on MTBE to derive the adaptation operations that must be implemented in a legacy model transformation when a NFR appears is presented. The approach derives semi-automatically the model transformations using execution traceability data and models differences. An example where access control property is integrated on a MDD system is introduced to demonstrate the usefulness of the tool to evolve model transformations.

Cyber-Physical Systems (CPSs) product lines appear in a wide range of applications of different domains (e.g., car's doors' windows, doors of a lift, etc.). The variability of these systems is large and as a result they can be configured into plenty of configurations. Testing each of the configurations can be time consuming as not only software has to be simulated, but also the hardware and the physical layer of the CPS, which is often modelled with complex mathematical models. Choosing the adequate test control strategy is critical when testing CPSs product lines. This paper presents a set of test control algorithms organized in an architecture of three layers (domain, application and simulation) for testing CPSs product lines. An illustrative example of a CPS product line is presented and three experiments are conducted to measure the performance of the proposed test control algorithms. We conclude that test scheduling and test suite minimization significantly help to reduce the overall test costs while preserving the test quality in CPSs product lines. In addition, we conclude that knowing the results of the previously tested configurations permits reducing the time for the detection of anomalous designs.

The number of cyber-physical systems is increasing considerably, being very usual in automation systems. Many cyber-physical systems must deal with variability issues to give response to the current market needs. The use of a plant model that simulates the system controlled by the embedded system might ease verification and validation activities, but building plant models manually for cyber-physical systems with many variants can be time consuming and error prone. This paper proposes a methodology that semi-automatically generates plant models in Simulink, with the main purpose of handling variability issues to validate highly configurable cyber-physical systems.

SUMMARY Nowadays, embedded systems are gaining importance. At the same time, the development of their software is increasing its complexity, having to deal with quality, cost, and time‐to‐market issues among others. With stringent quality requirements such as performance, early verification and validation become critical in these systems. In this regard, advanced development paradigms such as model‐driven engineering and software product line engineering bring considerable benefits to the development and validation of embedded system software. However, these benefits come at the cost of increasing process complexity. This work presents a process based on UML and MARTE for the analysis of embedded model‐driven product lines. It specifies the tasks, the involved roles, and the workproducts that form the process and how it is integrated in the more general development process. Existing tools that support the tasks to be performed in the process are also described. A classification of such tools and a study of traceability among them are provided, allowing engineering teams to choose the most adequate chain of tools to support the process. Copyright © 2012 John Wiley & Sons, Ltd.

Nowadays, embedded system development is increasing its complexity dealing with quality, cost and time-to-market among others. Quality attributes are an important issue to consider in embedded software development where time issues may be critical. Development paradigms such as Model Driven Development and Software Product Lines can be an adequate alternative to traditional software development and validation methods due to the characteristics of embedded systems. But for a proper validation and verification based on MARTE model analysis, all variability issues and critical quality attributes that take part in analysis must be properly modelled and managed. Therefore, a model analysis process for Model Driven Embedded Software Product Lines has been defined as some process lacks have been found.

Variability is a key aspect in software product lines. Functional variability has been largely studied as a way to obtain all the desired products for a line. Quality variability, less understood and more complex, has not received so much attention by researchers. However, different members of the line may require different levels of a quality attribute. The design phase is a good point to assure that quality attributes requirements are met within the product line so this means paying attention to software architecture evaluation during domain engineering. The quality evaluation in software product lines is much more complicated than in single-systems as products can require different quality levels and the product line can have variability on design that in turn affects quality. The evaluation of all the products of a line is very expensive. Thus, ways of reducing the evaluation efforts are necessary. Herein is presented a method for facilitating cost-effective quality evaluation of a product line taking into consideration variability on quality attributes.

Obtaining the appropriate architecture for a software system assures the long-term success of the product. Without a well defined and documented architecture is very hard or impossible to analyze and evaluate the quality of the product and the change impact. Thus it is very expensive to communicate to each stakeholder the information needed. Many existing approaches propose models based on a "closed" set of views to document architectures, such as "4+1" model, Siemens’ model, etcetera. Nevertheless, industrial practices in Small and Medium size Enterprises (SME) works informally, often even not documenting. From a practical experience where the 4+1 model was applied, it was noticed that there were views not proposed that were interesting, therefore a method was defined to select the views that really turn out useful to document in each case. To this end, the approach proposed by Clements et al [1] has been adapted.