The dependability assessment of critical systems must consider the emulation of transient faults, as they pose an important dependability threat for modern VLSI designs. FPGA-based systems are dominated by bit-flips in configuration memory (CM), which are relatively easy to emulate using partial runtime reconfiguration (RTR) FPGA fault injection (FFI) approaches. However, when FPGA is used as an ASIC prototyping platform, transient fault models representative of ASIC designs must be considered, such as bit-flips in sequential logic cells (Flip-Flops and on-chip RAM blocks). Existing RTR-FFI approaches do not adequately cover these faults, as they require the orchestrated manipulation of multiple CM bits for each target logic cell, and the location of these CM bits is unknown (not documented) for modern FPGA generations. This work experimentally formalises the mapping of the necessary CM bits, proposes an enhanced RTR-FFI methodology to emulate bit-flips in registers and on-chip RAMs of current-generation AMD Ultrascale+ FPGAs, and provides an upgraded publicly available open source FFI tool (BAFFI, https://gitlab.com/selene-riscv-platform/DAVOS ) supporting the proposed methodology. The validity of the proposed FFI approach is demonstrated by comparison with gate-level simulation-based fault injection in a case study of two soft-core processors (MC8051 and NOEL-V).

FPGA-based fault injection (FFI) is an indispensable technique for verification and dependability assessment of FPGA designs and prototypes. Existing FFI tools make use of Xilinx essential bits technology to locate the relevant fault targets in FPGA configuration memory (CM). Most FFI tools treat essential bits as black-box, while few of them are able to filter essential bits on the area basis in order to selectively target design components contained within the predefined Pblocks. This approach, however, remains insufficiently precise since the granularity of Pblocks in practice does not reach the smallest design components. This paper proposes an open-source FFI tool that enables much more fine-grained FFI experiments for Xilinx 7-series and Ultrascale+ FPGAs. By mapping the essential bits with the hierarchical netlist, it allows to precisely target any component in the design tree, up to an individual LUT or register, without the need for defining Pblocks (floorplanning). With minimal experimental effort it estimates the contribution of each DUT component into the resulting dependability features, and discovers weak points of the DUT. Through case studies we show how the proposed tool can be applied to different kinds of DUTs: from small-footprint microcontrollers, up to multicore RISC-V SoC. The correctness of FFI results is validated by means of RT-level and gate-level simulation-based fault injection.

The benchmarking of security tools is endeavored to determine which tools are more suitable to detect system vulnerabilities or intrusions. The analysis process is usually oversimplified by employing just a single metric out of the large set of those available. Accordingly, the decision may be biased by not considering relevant information provided by neglected metrics. This article proposes a novel approach to take into account several metrics, different scenarios, and the advice of multiple experts. The proposal relies on experts quantifying the relative importance of each pair of metrics towards the requirements of a given scenario. Their judgments are aggregated using group decision making techniques, and pondered according to the familiarity of experts with the metrics and scenario, to compute a set of weights accounting for the relative importance of each metric. Then, weight-based multi-criteria-decision-making techniques can be used to rank the benchmarked tools. The usefulness of this approach is showed by analyzing two different sets of vulnerability and intrusion detection tools from the perspective of multiple/single metrics and different scenarios.

The high complexity of new designs and time-to-market pressure have caused design reuse to be at the heart of the common semi-custom hardware design flow. Accordingly, current Electronic Design Automation (EDA) toolchains are developed to support a wide range of hardware description languages, third-party EDA tools, intellectual property cores, and implementation technologies and goals. However, the seamless integration of dependability requirements into such toolchains remains today an open challenge. This paper presents DAVOS, an EDA toolkit supporting assessment, verification, optimisation (design space exploration), and selection (benchmarking) processes for dependability-aware hardware implementations. This toolkit fully automates these processes with efficiency and flexibility in mind, so underlying implementation and analysis phases can be customized to consider alternative off-the-self languages, tools, components and technologies from a dependability perspective. Three different embedded processor models exemplify the design scenarios supported by DAVOS.

This paper studies the impact of logical synthesizers parameters on the performance, power-consumption, area (PPA) and dependability of HW implementations. Deducing optimal synthesis-parameter configurations attending to specific goals is challenging even for simple HW models. The proposal relies on fractional factorial design of experiments to minimize simulation-based fault-injection time. The set of synthesis parameters with an statistically significant impact on PPA and dependability goals is then deduced and regression models are generated to estimate such impact for any synthesis-parameter configuration. Optimal configurations are finally selected attending to specific implementation goals. The whole methodology is automated and applied onto the Xilinx XST synthesizer working on a simplex and TMR version of an enhanced Intel 8051 microcontroller model, but it can be potentially applied to any synthesizer and any HDL-based model. Results show that non-negligible benefits in terms of PPA and dependability can be obtained by simply tuning synthesizer parameters in a proper way.

Increasingly complex microcontroller designs for safety-relevant automotive systems require the adoption of new methods and tools to enable a cost-effective verification of their robustness. In particular, costs associated to the certification against the ISO26262 safety standard must be kept low for economical reasons. In this context, simulation-based verification using instruction set simulators (ISS) arises as a promising approach to partially cope with the increasing cost of the verification process as it allows taking design decisions in early design stages when modifications can be performed quickly and with low cost. However, it remains to be proven that verification in those stages provides accurate enough information to be used in the context of automotive microcontrollers. In this paper we analyze the existing correlation between fault injection experiments in an RTL microcontroller description and the information available at the ISS to enable accurate ISS-based fault injection.

Routing protocols are key elements for ad hoc networks. They are in charge of establishing routes between network nodes efficiently. Despite the interest shown by the scientific community and industry in converting the first specifications of ad hoc routing protocols in functional prototypes, aspects such as the resilience of these protocols remain generally unaddressed in practice. Tackling this issue becomes critical given the increasingly variety of accidental and malicious faults (attacks) that may impact the behaviour exhibited by ad hoc routing protocols. The main objective of this paper is to deepen in the methodological aspects concerning fault injection in routing protocols. As a result, we will design and implement a framework based on the injection of accidental and malicious faults to quantitatively evaluate their impact on routing protocols. This framework, called REFRAHN (Resilience Evaluation FRamework for Ad Hoc routiNg protocols), can be used to (i) reduce the uncertainty about the sources of perturbations in the deployment of ad hoc routing protocols, (ii) design fault-tolerant mechanisms that address and minimise such problems, and (iii) compare and select which is the routing protocol that optimises the performance and robustness of the network.

Routing protocols allow for the spontaneous formation of wireless multi-hop networks without dedicated infrastructure, also known as ad hoc networks. Despite significant technological advances, difficulties associated with the evaluation of ad hoc routing protocols under realistic conditions, still hamper their maturation and significant roll out in real world deployments. In particular, the resilience evaluation of ad hoc routing protocols is essential to determine their ability of keeping the routing service working despite the presence of changes, such as accidental faults or malicious ones (attacks). However, the resilience dimension is not always addressed by the evaluation platforms that are in charge of assessing these routing protocols. In this paper, we provide a survey covering current state-of-the-art evaluation platforms in the domain of ad hoc routing protocols paying special attention to the resilience dimension. The goal is threefold. First, we identify the most representative evaluation platforms and the routing protocols they have evaluated. Then, we analyse the experimental methodologies followed by such evaluation platforms. Finally, we create a taxonomy to characterise txperimental properties of such evaluation platforms.

Wireless Mesh networks (WMN) typically rely on proactive routing protocols to establish optimal communication routes between every pair of system nodes. These protocols integrate link-quality-based mechanisms to minimise the adverse effect of ambient noise on communications. This paper shows the limitations existing in such mechanisms by analysing the impact of ambient noise on three state-of-the-art proactive routing protocols: OLSR, B.A.T.M.A.N and Babel. As will be shown, the lack of context-awareness in their link-quality mechanisms prevents the protocols from adjusting their behaviour according to persistent levels of ambient noise, which may vary along the time. Consequently, they cannot minimise the impact of such noise on the availability of network routes. This issue is very serious for a WMN since the loss communication links may strongly increase the convergence time of the network. An adaptive extension to studied link-quality-based mechanisms is proposed to avoid the loss of communication links in the presence of high levels of ambient noise. The effectiveness of the proposal is experimentally assessed, thus establishing a new method to reduce the impact of ambient noise on WMN.

Field-Programmable Gate Arrays (FPGA) have proven their value over time as final implementation targets. Their singular architecture renders them sensitive to a wide range of faults, specially to those causing multiple and non-simultaneous errors, that can result in silent data corruption and also in structural changes in the hardware implementation. This papers presents and tests an approach to enable the confident use of conventional (low-cost) FPGAs in hostile environments. The design combines spatial and temporal redundancy with partial dynamic reconfiguration to increase the resilience of designs. The goal is to tolerate the occurrence of single and multiple faults, even during the reconfiguration process of FPGAs, while minimizing the impact of the recovery process on system availability. Fault injection techniques are used to experimentally evaluate various features of the approach. Results are very promising and lead us to state that, although many research is still required, the old idea of self-repairing HW designs is closer today.

Ad hoc networks are gaining momentum in contexts of use where the rapid deployment of dynamic networks is critical. It is well-known that these networks are threatened by a variety of accidental and malicious faults. Despite this fact, there exist few critical studies quantifying and analysing the impact of faults on networks' performance, resilience and energy consumption. This practical experience report studies the effect of eleven different types of perturbations (including malicious and accidental faults) on different versions of a well-known ad hoc routing protocol implementation, named olsrd. Experimentation also includes the assessment of the effectiveness of fault prevention (encryption) and detection (watchdog) mechanisms available for the considered protocols versions. During the analysis, emphasis is placed on the influence that context of use has on the interpretation of results. In that sense, reported experimentation is particularised to the domain of wireless mesh networks and mobile ad hoc networks.

The integration of Commercial-Off-The-Shelf (COTS) components in software has reduced time-to-market and production costs, but selecting the most suitable component, among those available, remains still a challenging task. This selection process, typically named benchmarking, requires evaluating the behaviour of eligible components in operation, and ranking them attending to quality characteristics. Most existing benchmarks only provide measures characterising the behaviour of software systems in absence of faults ignoring the hard impact that both accidental and malicious faults have on software quality. However, since using COTS to build a system may motivate the emergence of dependability issues due to the interaction between components, benchmarking the system in presence of faults is essential. The recent ISO/IEC 25045 standard copes with this lack by considering accidental faults when assessing the recoverability capabilities of software systems. This paper proposes a dependability benchmarking approach to determine the impact that faults (noted as disturbances in the standard) either accidental or malicious may have on the quality features exhibited by software components. As will be shown, the usefulness of the approach embraces all evaluator profiles (developers, acquirers and third-party evaluators) identified in the ISO/IEC 25000 "SQuaRE" standard. The feasibility of the proposal is finally illustrated through the benchmarking of three distinct software components, which implement the OLSR protocol specification, competing for integration in a wireless mesh network.

Ad hoc routing protocols are threatened by a variety of accidental and malicious faults that limit their use. Although a number of well-known strategies exist to enhance the performance and resilience of such type of protocols, their final effectiveness strongly relies on the usage of appropriate protocol configuration parameters. This paper investigates how to parameterise ad hoc routing protocols to combine high performance, with acceptable levels of resilience and low consumption of resources. The research places the spotlight on olsrd, an ad hoc proactive routing protocol able to run on real devices and deploy challenge-response authentication, packet signature and fault tolerance strategies at runtime. The reported practical experience is carried out in different ad hoc networking contexts integrating different types of devices, thus checking the influence that mobility of nodes and device resource constraints have on the parameterisation of the different protocol features considered.

The increasing emergence of mobile computing devices seamlessly providing wireless communication capabilities opens a wide range of new application domains for ad hoc networks. However, the sensitivity of ad hoc routing protocols to malicious faults (attacks) limits in practice their confident use in commercial products. This requires not only practical means to enforce the security of these protocols, but also approaches to evaluate their behaviour in the presence of attacks. Our previous contribution to the evaluation of ad hoc networks has been focused on the definition of an approach for injecting grey hole attacks in real (non-simulated) ad hoc networks. This paper relies on the use of this methodology to evaluate (i) three different implementations of a proactive ad hoc routing protocol, named OLSR, and (ii) two ad hoc routing protocols of different nature, one proactive (OLSR) and one reactive (AODV). Reported results have proven useful to extend the applicability of attack injection methodologies for evaluation beyond the mere assessment of the robustness of ad hoc routing protocols.

Current practices for the design and deployment of hardware redundancy techniques in embedded systems remain in practice specific (defined on a case-per-case basis) and mostly manual. This paper addresses the challenging problems of engineering fault tolerance mechanisms in a generic way and providing suitable tools for coping with their deployment. This approach relies on metaprogramming to specify fault tolerance mechanisms and open compilers to automatically deploy such mechanisms on the selected hardware core. Our previous research has already shown the suitability of this approach for the generic design and automatic deployment of NMR strategies. In this paper, we explore the usefulness of the idea in the context of information redundancy. The main contribution is the development of a metaprogram for the provision of ECC-based fault tolerance in data storage elements (registers and memory modules). It is also shown to what extend such metaprogram can be useful for improving the reliability of communications between HW cores in embedded systems.

Ad hoc networks constitute a quick and cheap alternative to provide communications when deploying a fixed infrastructure could result prohibitive in terms of either time or money. Although the unique properties of ad hoc networks make them very sensitive to malicious faults attacks), most of current research has focused on improving their routing capabilities, thus widening their application domains, and very little attention has been devoted to their dependability. Accordingly, the confident use of this technology requires the development of new techniques and tools to assess the robustness of such networks in presence of attacks. This paper deals with this challenging goal by proposing an attack injection approach based on real ad hoc networks as experimental platform. Experiments show the feasibility of the proposed approach and identify a large number of possibilities to increase our knowledge on how real ad hoc networks behave in practice.

Hardware fault tolerance is a requirement even for noncritical applications, since unexpected failures may damage the reputation of manufacturers and limit the acceptance of their products. However, current practices for the design and deployment of hardware redundancy techniques remain in practice specific (defined on a case-per-case basis) and mostly manual. This paper addresses the challenging problems of (i) engineering NMR strategies in a generic way, and (ii) automating their deployment. This approach relies on metaprogramming to specify NMR mechanisms and open compilers to automatically deploy such mechanisms on the selected hardware core. Fault injection complements that approach by providing the means to (i) determine the best core for replication, and (ii) check the effectiveness of the deployed NMR strategy. A PIC microcontroller is used as case study to exemplify the approach and show its feasibility.

Advances in semiconductor technologies are greatly increasing the likelihood of fault occurrence in deep-submicrometer manufactured VLSI systems. The dependability assessment of VLSI critical systems is a hot topic that requires further research. Field-programmable gate arrays (FPGAs) have been recently pro posed as a means for speeding-up the fault injection process in VLSI systems models (fault emulation) and for reducing the cost of fixing any error due to their applicability in the first steps of the development cycle. However, only a reduced set of fault models, mainly stuck-at and bit-flip, have been considered in fault emulation approaches. This paper describes the procedures to inject a wide set of faults representative of deep-submicrometer technology, like stuck-at, bit-flip, pulse, indetermination, stuck-open, delay, short, open-line, and bridging, using the best suitable FPGA- based technique. This paper also sets some basic guidelines for comparing VLSI systems in terms of their availability and safety, which is mandatory in mission and safety critical application contexts. This represents a step forward in the dependability benchmarking of VLSI systems and towards the definition of a framework for their evaluation and comparison in terms of performance, power consumption, and dependability.

Advances in circuitry integration increase the probability of occurrence of transient faults in VLSI systems. A confident use of these systems requires the study of their behaviour in the presence of such faults. This study can be conducted using model-based fault injection techniques. In that context, field-programmable gate arrays (FPGAs) offer a great promise by enabling those techniques to execute models faster. This paper focuses on how run-time reconfiguration techniques can be used for emulating the occurrence of transient faults in VLSI models. Although the use of FPGAs for that purpose has been restricted so far to the well-known bit-flip fault model, recent studies in fault representativeness point out the need of considering a wider set of faults modelling aspects like delays, indeterminations and pulses. Therefore, the main goal of this study is to analyse the different alternatives that FPGAs offer for the emulation of these faults while greatly decreasing the time devoted to models execution

A confident use of deep-submicron VLSI systems requires the study of their behaviour in the presence of faults, which has been traditionally conducted via model-based fault injection techniques. Although field-programmable gate arrays (FPGAs) allows for a fast execution of models, its use to emulate the occurrence of permanent faults in VLSI models has been restricted so far to the well-known stuck-at fault model. Recent studies in fault representativeness point out the need of considering a wider set of faults modelling aspects like delays or short circuits. This paper presents new and different alternatives for the emulation of permanent faults. Several experiments have been performed using an automated tool that allows for the injection of all the studied fault models. Results from these experiments show both the feasibility of the proposed approach, and the time saving achieved by executing the models on FPGAs

A confident use of deep submicron VLSI systems requires the study of their behaviour in the presence of faults. Field-programmable gate arrays (FPGAs) are being used to conduct this study by means of fault injection in a very fast way. However, FPGA-based fault injection tools are mainly focused on classical faults like stuck-at and bit-flip, and do not cover fault models related to new semiconductor technologies like delay, pulse, stuck-open, short, open-line, bridging, and indetermination. Moreover, these tools usually require a deep fault injection background to use them. This paper presents FADES, a tool for the early and fast dependability evaluation of VLSI systems. FADES is able to inject the whole set of considered faults and also enables non-skilled users to assess their systems' dependability. The main advantages and drawbacks of FADES are reported, and some open challenges for further research are identified