Abstract Predictive Mutation Testing (PMT) has emerged as a promising technique for reducing the high computational cost of traditional mutation testing. This is achieved by using predictive models to estimate mutant behavior without actually requiring their execution. However, despite its growing interest, PMT is still a young and evolving field and, therefore, its experimental studies remain particularly vulnerable to various methodological threats that may compromise the validity, comparability, and reproducibility of results. This paper aims to identify and examine key methodological threats in prior PMT studies that not only affect their validity, but may also limit the technique’s ability to reach its full predictive potential. For each of the eight identified threats, spanning the main stages of the PMT workflow, we describe its nature, analyze how it has been addressed (or overlooked) in previous work, and offer a validation list of recommended practices. We further conduct an empirical validation to substantiate the impact of these threats. In addition, based on the gaps identified in this analysis, we outline several open challenges that remain unexplored in the field, such as the lack of standardization in dataset sharing, the prevalence of Java-centric studies with method-level operators, and the challenges posed by class imbalance and project heterogeneity. This work contributes to strengthening the methodological soundness of PMT research, promoting more meaningful cross-study comparisons, and encouraging the adoption of practices that foster reproducibility, scalability, and real-world applicability.

Road safety is a critical global issue, with traffic accidents causing substantial human and economic losses annually. Advanced Driver Assistance Systems (ADAS) have emerged as an effective solution, leveraging technologies such as computer vision and artificial intelligence to enhance driving safety. However, the high costs and complexity of these systems limit their widespread adoption. This paper presents a cost-effective ADAS prototype utilizing vision-based and deep learning technologies to monitor and maintain safe following distances between vehicles. The performance of the system is evaluated using standard datasets and real-world scenarios, demonstrating adaptability across diverse driving conditions. Additionally, the feasibility of deploying the system on low-resource devices, such as Raspberry Pi and Radxa Zero, is analyzed, highlighting its potential for practical applications.

Efficiently comprehending vast volumes of text, particularly in scientific literature, is a formidable challenge for researchers. To alleviate this burden, this paper investigates the application of topic modeling and extractive summarization techniques to analyze and distill valuable insights from scientific documents. Focusing on the NeurIPS dataset, which encompasses a diverse array of scientific papers, we employ techniques such as Latent Dirichlet Allocation (LDA) and probabilistic Latent Semantic Analysis (pLSA) to categorize documents based on their content. Subsequently, we implement a domain-specific extractive summarizer that leverages Convolutional Neural Networks (CNNs) for feature extraction and semantic segmentation. The summarizer is trained to identify and rank sentences based on relevance, using binarized ROUGE-2 scores as a metric. Our results demonstrate the effectiveness of these methods in condensing scientific documents, facilitating quicker information retrieval and understanding. The paper also compares the performance of LDA and pLSA, highlighting LDA’s superiority in topic coherence and generalizability. By leveraging these methods, we aim to demonstrate their efficacy in processing large volumes of text, facilitating enhanced information retrieval and understanding in the realm of scientific literature.

The global aging population has led, over the last few years, to the proliferation of Ambient Assisted Living (AAL) systems. These include Internet of Things (IoT)-based systems for monitoring, activity recognition, fall detection and even indoor location systems. The wide variety of existing systems highlights the interest in AAL as a tool to improve the life of elderly and dependent people. However, such variety is also an issue for data integration, which in turn challenges the development of more advanced functionalities for their care. To solve this issue, this paper proposes a model-driven approach that facilitates the creation of information flows to combine several existing IoT solutions for AAL systems. For this purpose, we present a metamodel that allow to represent information flows for heterogeneous AAL systems. In addition, a graphical tool to facilitate the design of such flows along with a mechanism to automatically transform the models into executable code is also provided. Finally, a use case oriented to monitoring elderly and dependent people routine is presented. Thus, we contribute to facilitating the integration of heterogeneous applications based on information flows to build more complex systems that improve real-time monitoring and care of the elderly and dependent people.

Automated test case generation has proven to be useful to reduce the usually high expenses of software testing. However, several studies have also noted the skepticism of testers regarding the comprehension of generated test suites when compared to manually designed ones. This fact suggests that involving testers in the test generation process could be helpful to increase their acceptance of automatically-produced test suites. In this paper, we propose incorporating interactive readability assessments made by a tester into EvoSuite, a widely-known evolutionary test generation tool. Our approach,InterEvo-TR, interacts with the tester at different moments during the search and shows different test cases covering the same coverage target for their subjective evaluation. The design of such an interactive approach involves a schedule of interaction, a method to diversify the selected targets, a plan to save and handle the readability values, and some mechanisms to customize the level of engagement in the revision, among other aspects. To analyze the potential and practicability of our proposal, we conduct a controlled experiment in which 39 participants, including academics, professional developers, and student collaborators, interact withInterEvo-TR. Our results show that the strategy to select and present intermediate results is effective for the purpose of readability assessment. Furthermore, the participants’ actions and responses to a questionnaire allowed us to analyze the aspects influencing test code readability and the benefits and limitations of an interactive approach in the context of test case generation, paving the way for future developments based on interactivity.

Abstract Mutation testing exploits artificial faults to measure the adequacy of test suites and guide their improvement. It has become an extremely popular testing technique as evidenced by the vast literature, numerous tools, and research events on the topic. Previous survey papers have successfully compiled the state of research, its evolution, problems, and challenges. However, the use of mutation testing in practice is still largely unexplored. In this paper, we report the results of a thorough study on the use of mutation testing in GitHub projects. Specifically, we first performed a search for mutation testing tools, 127 in total, and we automatically searched the GitHub repositories including evidence of their use. Then, we focused on the top ten most widely used tools, based on the previous results, and manually revised and classified over 3.5K GitHub active repositories importing them. Among other findings, we observed a recent upturn in interest and activity, with Infection (PHP), PIT (Java) and Humbug (PHP) being the most widely used mutation tools in recent years. The predominant use of mutation testing is development, followed by teaching and learning, and research projects, although with significant differences among mutation tools found in the literature—less adopted and largely used in teaching and research—and those found in GitHub only—more popular and more widely used in development. Our work provides a new and encouraging perspective on the state of practice of mutation testing.

Abstract Software testing is a complex and costly stage during the software development lifecycle. Nowadays, there is a wide variety of solutions to reduce testing costs and improve test quality. Focussing on test case generation, Dynamic Symbolic Execution (DSE) is used to generate tests with good structural coverage. Regarding test suite evaluation, Mutation Testing (MT) assesses the detection capability of the test cases by introducing minor localised changes that resemble real faults. DSE is however known to produce tests that do not have good mutation detection capabilities: in this paper, the authors set out to solve this by combining DSE and MT into a new family of approaches that the authors call Mutation‐Inspired Symbolic Execution (MISE). First, this known result on a set of open source programs is confirmed: DSE by itself is not good at killing mutants, detecting only 59.9% out of all mutants. The authors show that a direct combination of DSE and MT (naive MISE) can produce better results, detecting up to 16% more mutants depending on the programme, though at a high computational cost. To reduce these costs, the authors set out a roadmap for more efficient versions of MISE, gaining its advantages while avoiding a large part of its additional costs.

The Industry 4.0 (I4.0) is a paradigm settled down by the introduction of the Internet of things (IoT) into the production and manufacturing environment. I4.0 promotes the connection of physical items such as sensors, devices, and enterprise assets, to each other and to the Internet. The information that flows through these items is vital because it serves to make relevant decisions. One of the main features of I4.0 is its adaptability to the human needs, this means that the items included in the I4.0 network are heterogeneous and they are large in number. The majority of I4.0 papers, which are focused on testing, describe a specific system or part of the I4.0 network. We have not found any paper that undertakes the testing of multiple connected IoT devices that will receive, process, and make decisions according to the complex and real data that travel through the network. In this article, we present IoT-TEG (Test Event Generator) 4.0, which is based on the test event generator system IoT-TEG[1]. IoT-TEG 4.0 provides two new main contributions: the generation of test cases, which can include all the different types of data that the connected I4.0 devices under study can manage, and real-time testing. Additionally, its validation using real IoT programs is included and the results show that IoT-TEG 4.0 allows us to conduct tests that mimic real IoT system behaviors.

Test generation is a costly but necessary testing activity to increase the quality of software projects. Automated testing tools based on evolutionary computation principles constitute an appealing modern approach to support testing tasks. However, these tools still find difficulties to detect certain types of plausible faults in real-world projects. Besides, recent studies have shown that, in general, automatically-generated tests do not resemble those manually written and, consequently, testers are reluctant to adopt them. We observe two key issues, namely the opacity of the process and the lack of cooperation with the tester, currently hampering the acceptance of automated results. Based on these findings, we explore in this paper how the interaction between current tools and expert testers would help address the test case generation problem. More specifically, we identify a number of interaction opportunities related to the object-oriented test case design driven to boost their readability and detection power. Using EvoSuite as base implementation, we present a proof of concept focused on the possibility to integrate readability assessment of the most promising test suites into a genetic algorithm.

Organisations today are constantly consuming and processing huge amounts of data. Such datasets are often heterogeneous, making it difficult to work with them quickly and easily due to their format constraints or their disparate data structures. Therefore, being able to efficiently and intuitively work with such data to analyse them in real time to detect situations of interest as quickly as possible is a great competitive advantage for companies. Existing approaches have tried to address this issue by providing users with analytics or modelling tools in an isolated way, but not combining them as a onein- all solution. In order to fill this gap, we present MEdit4CEP-SP, a model-driven system that integrates Stream Processing (SP) and Complex Event Processing (CEP) technologies for consuming, processing and analysing heterogeneous data in real time. It provides domain experts with a graphical editor that allows them to infer and define heterogeneous data domains, while also modelling, in a user-friendly way, the situations of interest to be detected in such domains. These graphical definitions are then automatically transformed into code, which is deployed in the processing system at runtime. The alerts detected by the system, in real-time, allow users to react as quickly as possible, thus improving the decision-making process. Additionally, MEdit4CEP-SP provides persistence, storing these definitions in a NoSQL database to permit their reuse by other instances of the system. Further benefits of this system are evaluated and compared with other existing approaches in this paper.

Summary Performance bugs are known to be a major threat to the success of software products. Performance tests aim to detect performance bugs by executing the program through test cases and checking whether it exhibits a noticeable performance degradation. The principles of mutation testing, a well‐established testing technique for the assessment of test suites through the injection of artificial faults, could be exploited to evaluate and improve the detection power of performance tests. However, the application of mutation testing to assess performance tests, henceforth called performance mutation testing (PMT), is a novel research topic with numerous open challenges. In previous papers, we identified some key challenges related to PMT. In this work, we go a step further and explore the feasibility of applying PMT at the source‐code level in general‐purpose languages. To do so, we revisit concepts associated with classical mutation testing and design seven novel mutation operators to model known bug‐inducing patterns. As a proof of concept, we applied traditional mutation operators as well as performance mutation operators to open‐source C++ programs. The results reveal the potential of the new performance‐mutants to help assess and enhance performance tests when compared with traditional mutants. A review of live mutants in these programs suggests that they can induce the design of special test inputs. In addition to these promising results, our work brings a whole new set of challenges related to PMT, which will hopefully serve as a starting point for new contributions in the area.

Mutation testing is a well-known fault-based technique that has been applied to different domains as new technologies have appeared. Evolutionary Mutation Testing (EMT) finds mutants that are useful to produce new test cases. It uses evolutionary algorithms to reduce the number of mutants that are generated, keeping as many difficult to kill and stubborn mutants (strong mutants) as possible in the reduced set. Given the popularity of real-time systems, the MuEPL mutation system was developed for the Esper Event Processing Language (EPL), a query language aimed at the Internet of Things (IoT). In past work, EMT was integrated into MuEPL, and it reduced the cost of finding strong mutants in some EPL queries but not in others. This study takes a step forward by proposing and evaluating two metaheuristics for EMT that combine EMT and random selection: one which bootstraps the hall of fame with a random subset (Bootstrapped EMT), and one which falls back to random selection after a certain point (Inverse EMT). While BEMT is shown to outperform IEMT in most cases, BEMT has not managed to outperform EMT. An additional experiment studies the impact of low-quality mutation operators in the relative performance of BEMT, IEMT and plain EMT. Results suggest that the MuEPL RRO operator was the reason for the poor performance of EMT in some scenarios.

The Internet of Things (IoT) is a popular paradigm which has been applied to different areas, such as smart cities, medicine, or business processes. One of the main drawbacks of an IoT system is the amount of information it has to manage and to monitor. This information arrives as events that need to be processed in real-time in order to make correct decisions. The event processing languages (EPLs) were designed to handle this information by defining event patterns which describe relevant situations to be detected and filter the information. In the majority of the relevant situations to detect, the events have a specific behavior which must be analyzed not only to define the event patterns but also to simulate them to test the IoT system. Moreover, in several situations it is quite difficult to obtain test events with specific values: adverse environment conditions, rise or fall in blood pressure, heart attack, falls, among others. In this paper, we introduce a complete study of falls as relevant situations; we show an analysis of the fall-involved events of two types of falls based on an IoT prototype, the event patterns to detect the falls and their test using the IoT-test event generator (IoT-TEG) tool. The fall analysis has highlighted the necessity to improve IoT-TEG with a new functionality which allows defining the desired conduct by defining behavior rules.

Summary Mutation testing is a testing technique that has been applied successfully to several programming languages. Despite its benefits for software testing, the high computational cost of mutation testing has kept it from being widely used. Several refinements have been proposed to reduce its cost by reducing the number of generated mutants; one of those is evolutionary mutation testing (EMT). Evolutionary mutation testing aims at generating a reduced set of mutants with an evolutionary algorithm, which searches for potentially equivalent and difficult to kill mutants that help improve the test suite. Evolutionary mutation testing has been evaluated in two contexts so far, ie, web service compositions and object‐oriented C++ programmes. This study explores its performance when applied to event processing language queries of various domains. This study also considers the impact of the test data, since a lack of events or the need to have specific values in them can hinder testing. The effectiveness of evolutionary mutation testing with the original test data generators and the new internet of things test event generator tool is compared in multiple case studies.

This paper presents an exact search-based technique to minimize test suites while maintaining their mutation coverage. The minimization of test suites is a hard problem whose solution is important both to reduce the cost of mutation testing and to precisely assess the quality of existing test suites. This problem can be addressed with Search-Based Software Engineering (SBSE) techniques, including metaheuristics and exact techniques. We have applied Integer Linear Programming (ILP) as an exact technique to reduce the effort of testing with very promising results. Our technique can be adapted to different formalisms but this paper focuses on testing WS-BPEL compositions, as it poses several interesting problems. Despite the fact that web service compositions are relatively small, as they just orchestrate web services, their execution can be very expensive because the deployment and execution of web services, and the underlying infrastructure, are not trivial. Therefore, although test suites for the compositions themselves are also usually small, it is fundamental to reduce, as much as possible and without losing coverage, their size.

In this paper, we propose a technique to optimize dynamic communication network topologies. Due to recent developments in virtualization technologies like hardware virtualization, software defined networking (SDN), and network function virtualization (NFV), a completely new way of automated network optimization becomes possible. We propose a technique to enable hybrid evolutionary dynamic optimization of networking infrastructures based on changes to its topology or state. Networks have numerous, partially competing, attributes, which might be subject to optimization. Here, we focus exemplarily on the problem of finding the optimal amount and position of monitors in a computer network in order to monitor the whole network traffic. This dynamic monitor selection problem is generalizable to the well-known NP-complete vertex cover problem. Experiments are conducted on different real-world networks using an evolutionary search heuristic. We studied the behavior of the proposed approach using 3 different change levels of the problem instances. Experimental results show that our proposed approach provides network configurations having a sufficiently high quality in reasonable time for all problem instances and change levels. Thus, using the proposed technique may be one helpful step towards an automated self-adapting network optimization.

For software quality assurance, many safety-critical industries appeal to the use of dynamic testing and structural coverage criteria. However, there are reasons to doubt the adequacy of such practices. Mutation testing has been suggested as an alternative or complementary approach but its cost has traditionally hindered its adoption by industry, and there are limited studies applying it to real safety-critical code. This paper evaluates the effectiveness of state-of-the-art mutation testing on safety-critical code from within the U.K. nuclear industry, in terms of revealing flaws in test suites that already meet the structural coverage criteria recommended by relevant safety standards. It also assesses the practical feasibility of implementing such mutation testing in a real setting. We applied a conventional selective mutation approach to a C codebase supplied by a nuclear industry partner and measured the mutation score achieved by the existing test suite. We repeated the experiment using trivial compiler equivalence (TCE) to assess the benefit that it might provide. Using a conventional approach, it first appeared that the existing test suite only killed 82% of the mutants, but applying TCE revealed that it killed 92%. The difference was due to equivalent or duplicate mutants that TCE eliminated. We then added new tests to kill all the surviving mutants, increasing the test suite size by 18% in the process. In conclusion, mutation testing can potentially improve fault detection compared to structural-coverage-guided testing, and may be affordable in a nuclear industry context. The industry feedback on our results was positive, although further evidence is needed from application of mutation testing to software with known real faults.

Mutation testing is a method used to assess and improve the fault detection capability of a test suite by creating faulty versions, called mutants, of the system under test. Evolutionary Mutation Testing (EMT), like selective mutation or mutant sampling, was proposed to reduce the computational cost, which is a major concern when applying mutation testing. This technique implements an evolutionary algorithm to produce a reduced subset of mutants but with a high proportion of mutants that can help the tester derive new test cases (strong mutants). In this paper, we go a step further in estimating the ability of this technique to induce the generation of test cases. Instead of measuring the percentage of strong mutants within the subset of generated mutants, we compute how much the test suite is actually improved thanks to those mutants. In our experiments, we have compared the extent to which EMT and the random selection of mutants help to find missing test cases in C++ object-oriented systems. We can conclude from our results that the percentage of mutants generated with EMT is lower than with the random strategy to obtain a test suite of the same size and that the technique scales better for complex programs.

In this paper, we propose a hybrid evolutionary algorithm (EA) for the optimization of efficient monitoring in dynamic communication networks. The first step towards improving communication infrastructures is gathering information about the current situation. One part of collecting this information is to implement an adequate monitoring in the network, i.e., the optimal positions and amount of monitoring devices, in order to analyze communication flows.

Summary Mutation testing is an effective but costly testing technique. Several studies have observed that some mutants can be redundant and therefore removed without affecting its effectiveness. Similarly, some mutants may be more effective than others in guiding the tester on the creation of high‐quality test cases. On the basis of these findings, we present an assessment of C++ class mutation operators by classifying them into 2 rankings: the first ranking sorts the operators on the basis of their degree of redundancy and the second regarding the quality of the tests they help to design. Both rankings are used in a selective mutation study analysing the trade‐off between the reduction achieved and the effectiveness when using a subset of mutants. Experimental results consistently show that leveraging the operators at the top of the 2 rankings, which are different, lead to a significant reduction in the number of mutants with a minimum loss of effectiveness.

Stream-based data processing systems, such as Complex Event Processing or data stream mining systems, may be composed of several components which may be implemented in various programming languages. In distributed scenarios, computer networks typically represent important bottlenecks. However, the performance of data exchange in local contexts may be as important as the performance of data exchange via computer networks. Local programming language barriers may represent important bottlenecks for components that are located on the same computer system. In distributed scenarios, it may be beneficial to relocate components on a single physical host for exploiting the higher local data throughput. The properties of stream-based systems pose challenges like high throughput requirements but also open up optimization potential such as leveraging batched transfers. We performed an experimental analysis of ways for bridging local programming language barriers using the examples of C, Java, and Python and analyzed the impact of batched forwarding. While local data exchange can be expected to offer a higher throughput than exchange across networks, our results show that batch forwarding can increase the local throughput by factors of up to 47.6 and we measured net throughputs up to 39.5 Gbps.

We present an approach for flexible distributed remote packet capturing with additional self-adaptivity and cooperation capabilities. Such techniques are needed for gaining comprehensive insight into large computer networks. With our system it is possible to operate multiple distributed remote packet capturing sensors from arbitrary locations. Advanced features like self-adaptivity or the cooperative use of sensors help to increase the performance. Empirical results obtained with a prototype indicate that our approach is efficient and allows capturing of traffic with speeds in the order of magnitude of Gigabit Ethernet. Furthermore, our approach integrates with and enables the use of existing packet processing applications.

While network traffic acquisition and processing is typically done with languages like C that allow low-level hardware access and optimizations, languages like Java and their ecosystems aim at easing complex tasks. With a combination of both, strengths can be combined such that more powerful and versatile network traffic processing systems can be engineered. However, while approaches using languages like C are evolved and optimized, network traffic acquisition and processing with the Java Virtual Machine (JVM) is not equivalently optimized and benefits from employing JVM-based languages are not fully exploited yet. We present methods for increasing the network traffic processing performance with the JVM and examples for leveraging dynamic capabilities via a domain specific language and self-adaptivity. We measured improvements by factors of up to 5.9 compared to the old approach and capture rates up to 4.46 million packets per second and could show that the prototype is capable to self-adapt based on performance constraints.

Summary Mutation testing is a successful testing technique based on fault injection. However, it can be very costly, and several cost‐reduction techniques for reducing the number of mutants have been proposed in the literature. Cost reduction can be aided by an analysis of mutation operators, but this requires the definition of specialized metrics. Several metrics have been proposed before, although their effectiveness and relative merits are not easy to assess. A step ahead in the evaluation of mutation‐reduction techniques would be a better metric to determine objectively the quality of a set of mutants with respect to a given test suite. This work introduces such a metric, which is naturally extended to mutation operators and may be used to reduce the number of mutants, particularly of equivalent mutants. Finally, a firm mutation analysis tool for WS‐BPEL service compositions is presented, and experimental results obtained by comparing different metrics on several compositions are presented. Copyright © 2014 John Wiley & Sons, Ltd.

Maintaining correctly operating computer networks is paramount for assuring properly operating information technology infrastructures. Thereby, the acquisition of network traffic data is one of the first steps. The acquisition of network traffic, however, can be very challenging, e.g., with respect to performance and resource requirements. In this paper, we analyze the possibility of using packet header data for efficiently partitioning live network traffic data into subsets with the aim on enabling distributed packet capturing and processing. The goal is to employ multiple sensors in a coordinated fashion such that the overall task is distributed among the participating sensors. Our results show that efficiently partitioning live network traffic based on packet header data is possible. Furthermore, we implemented a prototype of a distributed packet capturing system that achieves significantly higher capture rates than a single, uncoordinated sensor.

The nature of the requirements analysis problem, based as it is on uncertain and often inaccurate estimates of costs and effort, makes sensitivity analysis important. Sensitivity analysis allows the decision maker to identify those requirements and budgets that are particularly sensitive to misestimation. However, finding scalable sensitivity analysis techniques is not easy because the underlying optimization problem is NP-hard. This article introduces an approach to sensitivity analysis based on exact optimization. We implemented this approach as a tool, O ATSAC , which allowed us to experimentally evaluate the scalability and applicability of Requirements Sensitivity Analysis (RSA). Our results show that O ATSAC scales sufficiently well for practical applications in Requirements Sensitivity Analysis. We also show how the sensitivity analysis can yield insights into difficult and otherwise obscure interactions between budgets, requirements costs, and estimate inaccuracies using a real-world case study.

Event processing queries are intended to process continuous event streams. These queries are partially similar to traditional SQL queries, but provide the facilities to express rich features (e.g., pattern expression, sliding window of length and time). An error while implementing a query may result in abnormal program behaviors and lost business opportunities. Moreover, queries can be generated with unsanitized inputs and the structure of intended queries might be altered. Thus, a tester needs to test the behavior of queries in presence of malicious inputs. Mutation testing has been found to be effective to assess test suites quality and generating new test cases. Unfortunately, there is no effort to perform mutation testing of event processing queries. In this work, we propose mutation-based testing of event processing queries. We choose Event Processing Language (EPL) as our case study and develop necessary mutation operators and killing criteria to generate high quality event streams and malicious inputs. Our proposed operators modify different features of EPL queries (pattern expression, windows of length and time, batch processing of events). We develop an architecture to generate mutants for EPL and perform mutation analysis. We evaluate our proposed EPL mutation testing approach with a set of developed benchmark containing diverse types EPL queries. The evaluation results indicate that the proposed operators and mutant killing criteria are effective to generate test cases capable of revealing anomalous program behaviors (e.g., event notification failure, delay of event reporting, unexpected event), and SQL injection attacks. Moreover, the approach incurs less manual effort and can complement other testing approach such as random testing.