Fault- and attack injection are techniques used to measure dependability attributes of computer systems. An important property of such techniques is their efficiency in exploring the target system's fault- or attack space. As this space is generally very large, pre-injection analysis techniques may be used to effectively explore the space. In this paper, we study two such techniques proposed in the past, namelyinject-on-readandinject-on-write. Furthermore, we propose two new techniques callederror space pruning of signalsanderror space pruning of signals and portsand evaluate their efficiency in reducing the space needed to be explored by injection experiments. These techniques were integrated into MODIFI, a fault- and attack injector targeting Simulink models. To the best of our knowledge, we are the first to evaluate these pre-injection techniques for this kind of injector. The results of our evaluation of 11 Simulink models from the automotive domain and one from the avionics domain, show that the new proposed techniques reduce the fault- and attack space needed to be explored by about 27–49%. Using MODIFI, we then performed injection experiments on two automotive models, as well as an aero engine control model, while elaborating on the results obtained.

Abstract This paper proposes a reference model for defining security benchmarks for the safety assessment of Cooperative Driving Automation (CDA) applications. Our reference model provides a systematic approach to benchmark the resilience of CDA applications against malicious attacks through extensive system simulations. It enables the test repeatability and comparison of results across different implementations of CDA applications. In our approach, a benchmark is defined as a series of tests that expose the target system to specific attacks while recording its response. Using this model, we define a benchmark for evaluating the resilience of Cooperative Adaptive Cruise Control (CACC) algorithms against barrage jamming attacks targeting the physical layer of the IEEE 802.11p communication standard. We apply this benchmark to assess and compare the performance of four CACC algorithms: P1, Flatbed, Ploeg, and Consensus. The benchmark measures reveal that the Consensus algorithm demonstrates the highest resilience against jamming attacks, primarily due to its heavy reliance on onboard sensors and the use of sensor data from all other vehicles for decision-making. In contrast, the P1 algorithm, which depends mainly on vehicle-to-vehicle (V2V) communication, proves to be the most vulnerable. Furthermore, the results indicate that vehicles are most susceptible to jamming attacks during acceleration phases, making these periods critical for security evaluation. These findings validate the effectiveness of our benchmarking framework in identifying strengths and vulnerabilities of CACC algorithms under cyberattacks.

A remotely operated road vehicle (RORV) refers to a vehicle operated wirelessly from a remote location. In this paper, we report results from an evaluation of two safety mechanisms: safe braking and disconnection. These safety mechanisms are included in the control software for RORV developed by Roboauto, an intelligent mobility solutions provider. The safety mechanisms monitor the communication system to detect packet transmission delays, lost messages, and outages caused by naturally occurring interference as well as denial-of-service (DoS) attacks. When the delay in the communication channel exceeds certain threshold values, the safety mechanisms are to initiate control actions to reduce the vehicle speed or stop the affected vehicle safely as soon as possible. To evaluate the effectiveness of the safety mechanisms, we exposed the vehicle control software to various communication failures using a software-in-the-loop (SIL) testing environment developed specifically for this study. Our results show that the safety mechanisms behaved correctly for a vast majority of the simulated communication failures. However, in a few cases, we noted that the safety mechanisms were triggered incorrectly, either too early or too late, according to the system specification.

This paper presents ComFASE, a communication fault and attack simulation engine. ComFASE is used to identify and evaluate potentially dangerous behaviours of interconnected automated vehicles in the presence of faults and attacks in wireless vehicular networks. ComFASE is built on top of OM-NET++ (a network simulator) and integrates SUMO (a traffic simulator) and Veins (a vehicular network simulator). The tool is flexible in modelling different types of faults and attacks and can be effectively used to study the interplay between safety and cybersecurity attributes by injecting cybersecurity attacks and evaluating their safety implications. To demonstrate the tool, we present results from a series of simulation experiments, where we injected delay and denial-of-service attacks on wireless messages exchanged between vehicles in a platooning application. The results show how different variants of attacks influence the platooning system in terms of collision incidents.

Fault- and attack injection are techniques used to measure dependability attributes of computer systems. An important property of such injectors is their efficiency that deals with the time and effort needed to explore the target system's fault- or attack space. As this space is generally very large, techniques such as pre-injection analyses are used to effectively explore the space. In this paper, we study two such techniques that have been proposed in the past, namely inject-on-read and inject-on-write. Moreover, we propose a new technique called error space pruning of signals and evaluate its efficiency in reducing the space needed to be explored by fault and attack injection experiments. We implemented and integrated these techniques into MODIFI, a model-implemented fault and attack injector, which has been effectively used in the past to evaluate Simulink models in the presence of faults and attacks. To the best of our knowledge, we are the first to integrate these pre-injection analysis techniques into an injector that injects faults and attacks into Simulink models. The results of our evaluation on 11 vehicular Simulink models show that the error space pruning of signals reduce the attack space by about 30–43%, hence allowing the attack space to be exploited by fewer number of attack injection experiments. Using MODIFI, we then performed attack injection experiments on two of these vehicular Simulink models, a comfort control model and a brake-by-wire model, while elaborating on the results obtained.

In this work, we evaluate the safety of a platoon of four vehicles under jamming attacks. The platooning application is provided by Plexe-veins, which is a cooperative driving framework, and the vehicles in the platoon are equipped with cooperative adaptive cruise control controllers to represent the vehicles' behavior. The jamming attacks investigated are modeled by extending ComFASE (a Communication Fault and Attack Simulation Engine) and represent three real-world attacks, namely, destructive interference, barrage jamming, and deceptive jamming. The attacks are injected in the physical layer of the IEEE 802.11p communication protocol simulated in Veins (a vehicular network simulator). To evaluate the safety implications of the injected attacks, the experimental results are classified by using the deceleration profiles and collision incidents of the vehicles. The results of our experiments show that jamming attacks on the communication can jeopardize vehicle safety, causing emergency braking and collision incidents. Moreover, we describe the impact of different attack injection parameters (such as, attack start time, attack duration and attack value) on the behavior of the vehicles subjected to the attacks.

Container-based virtualization is gaining popularity in different domains, as it supports continuous development and improves the efficiency and reliability of run-time environments. Different techniques are proposed for monitoring the security of containers. However, there are no guidelines supporting the selection of suitable techniques for the tasks at hand. We aim to support the selection and design of techniques for monitoring container-based virtualization environments. : First, we review the literature and identify techniques for monitoring containerized environments. Second, we classify these techniques according to a set of categories, such as technical characteristic, applicability, effectiveness, and evaluation. We further detail the pros and cons that are associated with each of the identified techniques. As a result, we present CONSERVE, a multi-dimensional decision support framework for an informed and optimal selection of a suitable set of container monitoring techniques to be implemented in different application domains. A mix of eighteen researchers and practitioners evaluated the ease of use, understandability, usefulness, efficiency, applicability, and completeness of the framework. The evaluation shows a high level of interest, and points out to potential benefits.

Recent studies have shown that technology and voltage scaling are expected to increase the likelihood that particle-induced soft errors manifest as multiple-bit errors. This raises concerns about the validity of using single bit-flips in fault injection experiments aiming to assess the program-level impact of soft errors. The goal of this article is to investigate whether multiple-bit errors could cause a higher percentage of silent data corruptions (SDCs) compared to single-bit errors. Based on 2700 fault injection campaigns with 15 benchmark programs, featuring a total of 27 million experiments, our results show that single-bit errors in most cases either yield a higher percentage of SDCs compared to multiple-bit errors or yield SDC results that are very close to the ones obtained for the multiple-bit errors. Further, we find that only around 2 percent of the multiple-bit campaigns resulted in an SDC percentage that was more than 5 percentage points higher than that obtained for the corresponding single-bit campaigns. For most of these campaigns, the highest percentage of SDCs was obtained by flipping at most 3 bits. Based on our results, we also propose four techniques for error space pruning to avoid injection of multiple-bit errors that are either unlikely or infeasible to cause SDCs.

Manufacturers of automated systems and their components have been allocating an enormous amount of time and effort in R&D activities. This effort translates into an overhead on the V&V (verification and validation) process making it time-consuming and costly. In this paper, we present an ECSEL JU project (VALU3S) that aims to evaluate the state-of-the-art V&V methods and tools, and design a multi-domain framework to create a clear structure around the components and elements needed to conduct the V&V process. The main expected benefit of the framework is to reduce time and cost needed to verify and validate automated systems with respect to safety, cyber-security, and privacy requirements. This is done through identification and classification of evaluation methods, tools, environments and concepts for V&V of automated systems with respect to the mentioned requirements. To this end, VALU3S brings together a consortium with partners from 10 different countries, amounting to a mix of 25 industrial partners, 6 leading research institutes, and 10 universities to reach the project goal.

Recent studies have shown that technology and voltage scaling are expected to increase the likelihood that particle-induced soft errors manifest as multiple-bit errors. This raises concerns about the validity of using single bit-flips for assessing the impact of soft errors in fault injection experiments. The goal of this paper is to investigate whether multiple-bit errors could cause a higher percentage of silent data corruptions (SDCs) compared to single-bit errors. Based on 2700 fault injection campaigns with 15 benchmark programs, featuring a total of 27 million experiments, our results show that single-bit errors in most cases yields a higher percentage of SDCs compared to multiple-bit errors. However, in 8% of the campaigns we observed a higher percentage of SDCs for multiple-bit errors. For most of these campaigns, the highest percentage of SDCs was obtained by flipping at most 3 bits. Moreover, we propose three ways of pruning the error space based on the results.

ISA-level fault injection, i.e. the injection of bitflip faults in Instruction Set Architecture (ISA) registers and main memory words, is widely used for studying the impact of transient and intermittent hardware faults. ISA-level fault injection tools can be characterized by different properties such as repeatability, observability, reachability, intrusiveness, efficiency and controllability. This paper presents two preinjection analysis techniques that improve controllability and efficiency using object code analysis. To improve controllability, we propose a technique for identifying the type of data that is stored in a potential target location. This allows the user to selectively direct fault injections to addresses, data and/or control information. Experimental results show that the data type of 84-100% of the targets locations in 8 programs were successfully identified by this technique. The second technique improves efficiency by fault pruning, i.e., by avoiding injection of faults that is known a priori to be detected by the tested system. This technique leverage the fact that faults in certain bits in the program counter and the stack pointer are always detected by machine exceptions. We show that exclusion of these bits from the fault space could significantly prune the fault space and reduce the time it takes to conduct a fault injection campaign.