Fault- and attack injection are techniques used to measure dependability attributes of computer systems. An important property of such techniques is their efficiency in exploring the target system's fault- or attack space. As this space is generally very large, pre-injection analysis techniques may be used to effectively explore the space. In this paper, we study two such techniques proposed in the past, namelyinject-on-readandinject-on-write. Furthermore, we propose two new techniques callederror space pruning of signalsanderror space pruning of signals and portsand evaluate their efficiency in reducing the space needed to be explored by injection experiments. These techniques were integrated into MODIFI, a fault- and attack injector targeting Simulink models. To the best of our knowledge, we are the first to evaluate these pre-injection techniques for this kind of injector. The results of our evaluation of 11 Simulink models from the automotive domain and one from the avionics domain, show that the new proposed techniques reduce the fault- and attack space needed to be explored by about 27–49%. Using MODIFI, we then performed injection experiments on two automotive models, as well as an aero engine control model, while elaborating on the results obtained.

A remotely operated road vehicle (RORV) refers to a vehicle operated wirelessly from a remote location. In this paper, we report results from an evaluation of two safety mechanisms: safe braking and disconnection. These safety mechanisms are included in the control software for RORV developed by Roboauto, an intelligent mobility solutions provider. The safety mechanisms monitor the communication system to detect packet transmission delays, lost messages, and outages caused by naturally occurring interference as well as denial-of-service (DoS) attacks. When the delay in the communication channel exceeds certain threshold values, the safety mechanisms are to initiate control actions to reduce the vehicle speed or stop the affected vehicle safely as soon as possible. To evaluate the effectiveness of the safety mechanisms, we exposed the vehicle control software to various communication failures using a software-in-the-loop (SIL) testing environment developed specifically for this study. Our results show that the safety mechanisms behaved correctly for a vast majority of the simulated communication failures. However, in a few cases, we noted that the safety mechanisms were triggered incorrectly, either too early or too late, according to the system specification.

This paper presents ComFASE, a communication fault and attack simulation engine. ComFASE is used to identify and evaluate potentially dangerous behaviours of interconnected automated vehicles in the presence of faults and attacks in wireless vehicular networks. ComFASE is built on top of OM-NET++ (a network simulator) and integrates SUMO (a traffic simulator) and Veins (a vehicular network simulator). The tool is flexible in modelling different types of faults and attacks and can be effectively used to study the interplay between safety and cybersecurity attributes by injecting cybersecurity attacks and evaluating their safety implications. To demonstrate the tool, we present results from a series of simulation experiments, where we injected delay and denial-of-service attacks on wireless messages exchanged between vehicles in a platooning application. The results show how different variants of attacks influence the platooning system in terms of collision incidents.

Fault- and attack injection are techniques used to measure dependability attributes of computer systems. An important property of such injectors is their efficiency that deals with the time and effort needed to explore the target system's fault- or attack space. As this space is generally very large, techniques such as pre-injection analyses are used to effectively explore the space. In this paper, we study two such techniques that have been proposed in the past, namely inject-on-read and inject-on-write. Moreover, we propose a new technique called error space pruning of signals and evaluate its efficiency in reducing the space needed to be explored by fault and attack injection experiments. We implemented and integrated these techniques into MODIFI, a model-implemented fault and attack injector, which has been effectively used in the past to evaluate Simulink models in the presence of faults and attacks. To the best of our knowledge, we are the first to integrate these pre-injection analysis techniques into an injector that injects faults and attacks into Simulink models. The results of our evaluation on 11 vehicular Simulink models show that the error space pruning of signals reduce the attack space by about 30–43%, hence allowing the attack space to be exploited by fewer number of attack injection experiments. Using MODIFI, we then performed attack injection experiments on two of these vehicular Simulink models, a comfort control model and a brake-by-wire model, while elaborating on the results obtained.

In this work, we evaluate the safety of a platoon of four vehicles under jamming attacks. The platooning application is provided by Plexe-veins, which is a cooperative driving framework, and the vehicles in the platoon are equipped with cooperative adaptive cruise control controllers to represent the vehicles' behavior. The jamming attacks investigated are modeled by extending ComFASE (a Communication Fault and Attack Simulation Engine) and represent three real-world attacks, namely, destructive interference, barrage jamming, and deceptive jamming. The attacks are injected in the physical layer of the IEEE 802.11p communication protocol simulated in Veins (a vehicular network simulator). To evaluate the safety implications of the injected attacks, the experimental results are classified by using the deceleration profiles and collision incidents of the vehicles. The results of our experiments show that jamming attacks on the communication can jeopardize vehicle safety, causing emergency braking and collision incidents. Moreover, we describe the impact of different attack injection parameters (such as, attack start time, attack duration and attack value) on the behavior of the vehicles subjected to the attacks.

Manufacturers of automated systems and their components have been allocating an enormous amount of time and effort in R&D activities. This effort translates into an overhead on the V&V (verification and validation) process making it time-consuming and costly. In this paper, we present an ECSEL JU project (VALU3S) that aims to evaluate the state-of-the-art V&V methods and tools, and design a multi-domain framework to create a clear structure around the components and elements needed to conduct the V&V process. The main expected benefit of the framework is to reduce time and cost needed to verify and validate automated systems with respect to safety, cyber-security, and privacy requirements. This is done through identification and classification of evaluation methods, tools, environments and concepts for V&V of automated systems with respect to the mentioned requirements. To this end, VALU3S brings together a consortium with partners from 10 different countries, amounting to a mix of 25 industrial partners, 6 leading research institutes, and 10 universities to reach the project goal.

This paper describes a framework for achieving node-level fault tolerance (NLFT) in distributed real-time systems. The objective of NLFT is to mask errors at the node level in order to reduce the probability of node failures and thereby improve system dependability. We describe an approach called lightweight NLFT where transient faults are masked locally in the nodes by time-redundant execution of application tasks. The advantages of light-weight NLFT is demonstrated by a reliability analysis of an example brake-by-wire architecture. The results show that the use of light-weight NLFT may provide 55% higher reliability after one year and almost 60% higher MTTF, compared to using fail-silent nodes.

This paper presents an experimental evaluation of a prototype jet engine controller intended for unmanned aerial vehicles (UAVs). The controller is implemented with commercial off-the-shelf (COTS) hardware based on the Motorola MPC565 microcontroller. We investigate the impact of single event upsets (SEUs) by injecting single bit-flip faults into main memory and CPU registers via the Nexus on-chip debug interface of the MPC565. To avoid the injection of non-effective faults, automated pre-injection analysis of the assembly code was utilized. Due to the inherent robustness of the software, most injected faults were still non-effective (69.4%) or caused bounded failures having only minor effect on the jet engine (7.0%), while 20.1% of the errors were detected by hardware exceptions and 1.9% were detected by executable assertions in the software. The remaining 1.6% is classified as critical failures. A majority of the critical failures were caused by erroneous Booleans or type conversions involving Booleans.

This paper describes the design and evaluation of a robust integrator for software-implemented control systems. The integrator is constructed as a generic component in the Simulink design tool, and can thus be used for robust implementation of a wide range of control algorithms. The integrator is designed to support the fail-bounded failure model for transient bit-flips that may occur in the CPU, main memory and I/O circuits of a control system. In particular, it allows the control system to detect and recover from bit-flips that cause data errors. Robustness is achieved by sequentially executing duplicated integrator code on the same processor to support error detection, and through the use of a recovery buffer that allows a roll-back to the previous integrator state when an error is detected. The effectiveness of the robust integrator was evaluated through fault injection experiments with a PI controller, where single bit flips were injected inside the CPU of the control system. No violations of the fail-bounded model were observed in the experiments.

Time redundant execution of tasks and comparison of results is a well-known technique for detecting transient faults in computer systems. However, time redundancy is also capable of detecting permanent faults that occur during or between the executions of two task replicas, provided the faults affect the results of the two tasks in different ways. In this paper, we derive an expression for estimating the probability of detecting data errors generated by permanent faults with time redundant execution. The expression is validated experimentally by injecting permanent stuck-at faults into a multiplier unit of a microprocessor. We use the derived expression to show how tasks can be scheduled to improve the detection probability of errors generated by permanent faults. We also show that the detection capability of permanent faults is low for the Temporal Error Masking (TEM) technique (i.e. triplicated execution and voting to mask transient faults) and may not be increased by scheduling. Thus, we propose complementing TEM with special test tasks.

This paper addresses the issue of characterizing the respective impact of fault injection techniques. Three physical techniques and one software-implemented technique that have been used to assess the fault tolerance features of the MARS fault-tolerant distributed real-time system are compared and analyzed. After a short summary of the fault tolerance features of the MARS architecture and especially of the error detection mechanisms that were used to compare the erroneous behaviors induced by the fault injection techniques considered, we describe the common distributed testbed and test scenario implemented to perform a coherent set of fault injection campaigns. The main features of the four fault injection techniques considered are then briefly described and the results obtained are finally presented and discussed. Emphasis is put on the analysis of the specific impact and merit of each injection technique.

This paper presents an experimental evaluation of a brake-by-wire application that tolerates transient faults by temporal error masking. A specially designed real-time kernel that masks errors by triple time-redundant execution and voting executes the application on a fail-stop computer node. The objective is to reduce the number of node failures by masking errors at the computer node level. The real-time kernel always executes the application twice to detect errors, and ensures that a fail-stop failure occurs if there is not enough CPU-time available for a third execution and voting. Fault injection experiments show that temporal error masking reduced the number of fail-stop failures by 42% compared to executing the brake-by-wire task without time redundancy.

We present a new fault injection tool called GOOFI (Generic Object-Oriented Fault Injection). GOOFI is designed to be adaptable to various target systems and different fault injection techniques. The tool is highly portable between different host platforms since it relies on the Java programming language and an SQL compatible database. The current version of the tool supports pre-runtime software implemented fault injection and scan-chain implemented fault injection.

Systems that use f+1 computer nodes to tolerate f node failures ordinarily require that the computer nodes have strong failure semantics, i.e. a node should either produce correct results or no results at all. We show that this requirement can be relaxed for control applications, as control algorithms inherently compensate for a class of value failures. Value failures occur when an error escapes the error detection mechanisms in the computer node and an erroneous value is sent to the actuators of the control system. Fault injection experiments show that 89% of the value failures caused by bit flips in a CPU had no or minor impact on the controlled object. However, the experiments also show that 11% of the value failures had severe consequences. These failures were caused by bit flips affecting the state variables of the control algorithm. Another set of fault injection experiments showed that the percentage of value failures with severe consequences was reduced to 3% when the state variables were protected with executable assertions and best-effort recovery mechanisms.