The Test-of-Time Award recognizes two outstanding papers published 10 years ago at DSN, in the DSN proceedings (research track, practical experience report or tool papers), that have had a sustained and important impact on the theory and/or practice of dependable systems and networks computing research. DSN has several areas under its umbrella and with two awards there are conditions to recognize more than one area. In exceptional situations (not enough nominations), the time frame for awards can be extended to 10-12 years, and only one paper can be awarded, in this order.

Abstract Computer‐based systems are now expected to evolve during their service life to cope with changes of various nature, ranging from evolution of user needs, eg, additional features requested by users, to system configuration changes, eg, modifications in available hardware resources. When considering resilient embedded systems that must comply with stringent dependability requirements, the challenge is even greater, as evolution must not impair dependability attributes. Maintaining dependability properties when facing changes is, indeed, the exact definition of resilient computing. In this paper, we consider the evolution of systems with respect to their dependability mechanisms and show how such mechanisms can evolve with the system evolution, in the case of ROS, the robot operating system. We provide a synthesis of the concepts required for resilient computing using a component‐based approach. We particularly emphasize the process and the techniques needed to implement an adaptation layer for fault tolerance mechanisms. In the light of this analysis, we address the implementation of adaptive fault tolerance on ROS in 2 steps: Firstly, we provide an architecture to implement fault tolerance mechanisms in ROS, and secondly, we describe the actual adaptation of fault tolerance mechanisms in ROS. Beyond the implementation details given in the paper, we draw the lessons learned from this work and discuss the limits of this run‐time support to implement adaptive fault tolerance features in embedded systems.

Engineering interactive systems for safety critical applications such as in avionic digital cockpits (and more generally Graphical User interfaces) is a challenge from a dependability viewpoint. The dependability of the user interface and its related hardware and software components must be consistent with the criticality of the functions to be controlled and their required DAL levels. This paper proposes a stepwise refinement approach going from systematic identification of failure modes of these systems to their detection via formally defined assertions. The last steps of the approach present how the assertions can be included into the monitoring part of self-checking interactive components and how they can be deployed on an architecture compliant with the ARINC 653 specification, ensuring temporal and spatial segregation, thus detecting errors and preventing failures due to both physical and transient software faults. We present how these contributions have been applied to the Flight Control Unit Backup interactive application which is available in A380 interactive cockpits.

Automotive embedded systems are becoming increasingly complex. Therefore verification activities are paramount to ensure safety. ISO 26262 is the first standard specifically dedicated to automotive safety systems. This standard requires introducing fault injection (FI) from the very early phases of the development process. Our work aims at developing an approach that will help integrate FI in the whole development process in a continuous way, from system requirements to the verification and validation phase. In this paper, we concentrate on exploring the benefits of safety analyses for experimental validation of the system. We propose an analogy between FI during the pre-implementation phase with safety analyses that are of common use during system design. We finally illustrate this approach on a case study from the automotive domain.

Assessing the correct behavior of a given system at run-time can be achieved by monitoring its execution, and is complementary to off-line analysis such as static verification. In this work, we focus on run-time monitoring of system properties that include both causality and timing constraints, in distributed and time-constrained systems. Based on a description of a property that includes events and temporal constraints, expressed as a timed-arc Petri net, we show how to automatically transform it into a an executable and distributed monitoring engine. To that aim, we introduce a modification of the semantics of Petri nets to be able to execute it online on partial executions and distributed observation environments. We show how to use this formal framework to provide MINOTOR, a model-driven distributed monitoring system, describe its implementation and show its applicability on a transportation use-case.

The evolution of systems during their operational lifetime is becoming ineluctable. Dependable systems, which continuously deliver trustworthy services, must evolve in order to comply with changes having different origins, e.g. new fault tolerance requirements, or changes in available resources. These evolutions must not violate their dependability properties, which leads to the notion of resilient computing. This paper presents a methodology for developing adaptive fault tolerance mechanisms, from the design to the actual runtime reconfiguration, leveraging component-based middleware which enable fine-grained manipulation of software architectures.

Supervising a system in operation allows to detect a violation of system specification or temporal properties, and is the first step required by any reconfiguration mechanism. In this work, we focus on run-time verification of temporal system properties in distributed and real-time systems. Based on a description of a property that includes events and temporal constraints, expressed as an arc timed Petri net, we automatically derive a monitoring system responsible for checking this property. The proposed approach enables the distributed verification of system properties. Our contribution is twofold. On the theoretical side, we introduce a slight modification of the semantics of Petri nets to be able to execute it in partial executions and noisy observation environments. On the practical side, we show how to use this formal framework to provide a distributed and efficient monitoring system, and describe its current implementation.

Modern computer systems are increasing in complexity, spread, and scale in order to meet the diverse and sophisticated needs of the users. In the development of these systems, we inevitably use legacy codes and off-the shelf software as black box software in order to shorten the development time and lower the development cost. These systems are often connected via a network to utilize services provided by other systems, whereas services and network performance may change while in operation. We often need to change the specification and implementation of a system due to the changes of environments and users' requirements. In addition, threats caused by viruses and unauthorized accesses have to be properly removed. Therefore, modern computer systems inherently involve incompleteness of specifications and implementations and uncertainty of environments and requirements.

In the last few years, glass cockpits are being replaced by interactive cockpits to provide a higher level of integration of both command and information display. Due to their event driven nature, interactive systems offer more display and control capabilities but they require specific error detection and fault tolerance techniques to reach a high level of dependability. This paper proposes a model-based approach for adding fault tolerance mechanisms to interactive cockpits. While several mechanisms are considered and presented, the contribution is focused on the formal description of self-checking widgets, being the basis for interactive cockpits.

New automotive modular multi-layered software organization particularly favors use and interoperability of components-off-the-shelf. However, the integration of software components is error-prone, if their coordination is not rigorously controlled. The risk of failure is increased with the possibility to multiplex software components with heterogeneous levels of criticality, observability. Most of dependability mechanisms, today, address locally errors within each component or report them to further diagnosis services. Instead, we consider a global wrapping-based approach to deal with multilevel properties to be checked on the complete multilayered system at runtime. In this paper, we introduce a framework to design robust software, from analysis to implementation issues, and we illustrate the methodology on simple case study.

Abstract In this paper we present COSMOPEN, a reverse‐engineering tool optimized for the behavioural analysis of complex layered software. COSMOPENcombines cheap and non‐intrusive observation techniques with a versatile graph manipulation engine. By programming different graph manipulation scripts, the ‘focal length’ of our tool can be adapted to different abstraction levels. We illustrate how our tool can be used to extract high‐level behavioural models from a complex multi‐threaded platform (GNU/Linux, CORBA middleware). Copyright © 2009 John Wiley & Sons, Ltd.

This workshop summary gives a brief overview of the workshop on ldquoArchitecting Dependable Systemsrdquo held in conjunction with DSN 2008. The main aim of this workshop is to promote cross-fertilization between the software architecture and dependability communities. We believe that both of them will benefit from clarifying approaches that have been previously tested and have succeeded as well as those that have been tried but have not yet been shown to be successful.

The evolution of systems during operational lifetime is becoming a core assumption of the design. This is the case for resource constrained embedded systems. Such an evolution may be driven by environment or the execution context. The adequacy of the service delivery with respect to the current operational conditions depends on the ability to tune the software configuration accordingly. This is true for application services, but also for dependability services, in particular the fault tolerance software. This paper presents a design of fault tolerance software for its runtime adaptation. This design relies on a reflective framework and open component based software engineering (CBSE) techniques. We demonstrate in this paper the feasibility of adapting componentized fault tolerance at a meta-level of the application.

Error confinement technologies have proven their efficiency to improve software dependability. Such mechanisms usually require efficient error detectors to swiftly signal any misbehaviour. Real-time systems, due to their timing constraints, require a richer description of correct and/or erroneous states that includes timing aspects. This paper presents real-time error detectors that can be automatically generated from formal models of the expected behaviours of software applications. The considered specifications provide the means to define quantitative temporal constraints on the execution of the application. These detectors check at run-time that the current execution matches its specification. The paper contribution is twofold. Firstly, at the theoretical level, we provide a formal definition of the expected behaviour of such detectors, ensuring a predictable behaviour of the detector system. Secondly, at a practical level, we provide a description of the complete generation process, from the models to the code of the detector.

Web Services are commonly used to implement service oriented architectures/applications. Service-oriented applications are large-scale distributed applications, typically highly dynamic, by definition loosely coupled and often unstable due to the unreliability of Web Services, which can be moved, deleted, and are subject to various sources of failures. In this paper, we propose customizable fault-tolerance connectors to add fault-tolerance to unreliable Web Services, thus filling the gap between clients and Web Service providers. Connectors are designed by clients, providers or dependability experts using the original WSDL description of the service. These connectors insert detection actions (e.g. runtime assertions) and recovery mechanisms (based on various replications strategies). The connectors can use identical or equivalent available service replicas. The benefits of this approach are demonstrated experimentally.

The past decade has seen an increasing use of complex computer systems made of third party components to develop mission critical applications. To insure the dependability of those systems in a sound and maintainable manner, technologies are needed to add fault-tolerance mechanisms transparently, while maintaining efficiency, high coverage, and evolvability. In this paper, we present a generic framework that addresses this problem and can be used within current industrial software. Our proposal is based on a limited set of core concepts inspired from plant biology and meta-object protocols. It provides separation of concerns for the implementation of adaptive fault tolerance strategies, while maintaining a global inter-level perception of the system runtime behavior. We demonstrate its practicality by using it to control the non-determinism of a CORBA/UNIX system.

Drivers are becoming the larger part of operating systems kernels. Previous studies have shown that device drivers seem to be one of the most important sources of operating systems misbehavior. Their failure can have significant impact on the kernel and cause significant damages to the system as a whole. To objectively characterize the impact of faulty drivers, we have carried out a series of fault injection experiments. To conduct these experiments we have targeted the DPI (Driver Programming Interface) that implements the way device drivers interact with the kernel. Faults are injected on the parameters of these kernel core Junctions. This allows for the derivation of useful results about the failure modes induced and thus characterization of the robustness of a target kernel with respect to faulty drivers. The information gathered also enables to improve these interaction facilities.

The goal of this paper is to assess the value of simple features that are widely available in off-the-shelf CORBA and Java platforms for the implementation of fault-tolerance mechanisms in industry-grade systems. This work builds on knowledge gained at LAAS from previous work on the prototyping of reflective fault tolerant frameworks. We describe how we used the interception and state capture mechanisms that are available in CORBA and Java to implement a simple replication strategy on a small middleware-based system built upon GNU/Linux and JOrbacus. We discuss the benefits and the limits of the resulting system from a practical point of view.

This paper presents an approach for building SoftWareImplemented Fault Injection (SWIFI) tools for real-time systems starting from temporal logic specifications. Temporal logic formulas are provided describing the behavior of the main SWIFI mechanisms, both for the injection of faults and for the observation of the target system. Besides conventional fault injectors (i.e., that flip bits in memory and in system calls’ parameters), we have also considered fault injectors based on the notion of saboteur. Concerning the observation mechanisms, we have defined on-line software probes aimed at observing not only the failure modes of the system under test, but also the real-time behavior of both the application and the underlying operating system. The specifications of all such mechanisms are then translated into programs that are executed on-line by a virtual machine. This approach was applied to the development of MAFALDA-RT, a SWIFI tool for real-time systems. A case study exemplifies how the proposed approach allows for the SWIFI mechanisms to be defined and applied.

Th3r2 party software is now in reasingly used in systems with hhm dependability requirements. Thq evolution of system development raises new h czM55LcO in parti ular regarding thg implementation of faulttoleran e. As systems are often built of bla k-box omponents, some ru ial aspe ts of thczz behzz5 regarding repli ation annot be h cWM ThW is also true to some extent for open-sour e omponents as mastering thste internal behnal c is sometimes very tri ky (e.g. OS and ORBs). During thi last de ade refle tion ho emerged as a very fruitful paradigm for dis iplined management of non-fun tional aspe ts, among wh h fault-toleran e. In thcW paper we dis uss hs to apply refle tion to multi-layer systems for implementing faulttoleran e in an independent and prin ipled manner. We analyze thl onne tions between thw underlying assumptions of fault-toleran e strategies and different layers of a system. Based on thcW multi-layer analysis we shcz hc thz requirements of a family of repli ation algorithz an be addressed on a on rete arhcz ture, resulting in whWLchMWLchchhO5Lzchzc tion.

This paper presents research work performed on the development and the verification of dependable reflective systems based on MetaObject Protocols (MOPS). We describe our experience, we draw the lessons learned from both a design and a validation viewpoint, and we discuss some possible future trends on this topic. The main originality of this work relies on the combination of both design and validation issues for the development of reflective systems, which has led to the definition of a reflective framework for the next generation of fault-tolerant systems. This framework includes: 1) the specification of a MetaObject Protocol suited to the implementation of fault-tolerant systems and 2) the definition of a general test strategy to guide its verification. The proposed approach is generic and solves many issues related to the use and evolution of system platforms with dependability requirements. Two different instances of the specified MOP have been implemented in order to study the impact of the MOP architecture in the development of a reflective fault-tolerant system. As far as the test strategy is concerned, a different testing level is associated with each reflective mechanism defined in the MOP. For each testing level, we characterize the test objectives and the required test environments. According to this experience, several new research challenges are finally identified.

The objective of this work is to define, implement and illustrate a portable serialization technique for CORBA objects. We propose an approach based on reflection: through open compilers facilities the internal state of CORBA objects is obtained and transformed into a language independent format using CORBA mechanisms. This state can be restored and used by objects developed using different languages and running on different software platforms. A tool was developed and applied to a Chat application as a case study. The proposed technique is used to exchange state information between a C++ and a Java incarnation of this CORBA service. An observer tool enables the object state to be displayed and analyzed by the user. The applicability of this technique to various domains is discussed. Beyond the interest of language reflection, we finally advocate that operating system and middleware reflection would also be powerful concepts to extend the work presented in this paper.

We present the principles of multi-level reflection as an enabling technology for the design and implementation of adaptive fault tolerant systems. By exhibiting the structural and behavioral aspects of a software component, the reflection paradigm enables the design and implementation of appropriate non-functional mechanisms at a meta-level. The separation of concerns provided by reflective architectures makes reflection a perfect match for fault tolerance mechanisms. However, in order to provide the necessary and sufficient information for error detection and recovery, reflection must be applied to all system layers in an orthogonal manner. This is the main motivation behind the notion of multi-level reflection that is introduced. We describe the basic concepts of this new architectural paradigm, and illustrate them with concrete examples. We also discuss some practical work that has recently been carried out to start implementing the proposed framework.

The dependability of CORBA systems is a crucial issue for the development of today's distributed platforms and applications. This paper analyzes various techniques that can be applied to the dependability evaluation of CORBA systems. Due to the complexity of a middleware platform like CORBA and its various types of software components, experiments using several fault injection techniques are required to obtain comprehensive dependability benchmarks. To illustrate one of these techniques, we have applied fault injection at the communication level, targeting requests to major CORBA services, such as naming and events. Experiments have been carried out on a number of off-the-shelf implementations of CORBA. We present and discuss some of the results that we have obtained. They provide objective insights into the system's behaviour in the presence of faults, and are significant inputs for the selection of a candidate for a given application domain.

The commercial offer concerning microkernel technology constitutes an attractive alternative for developing operating systems to suit a wide range of application domains. However, the integration of commercial off-the-shelf (COTS) microkernels into critical embedded computer systems is a problem for system developers, in particular due to the lack of objective data concerning their behavior in the presence of faults. This paper addresses this issue by describing a prototype environment, called MAFALDA (Microkernel Assessment by Fault injection AnaLysis and Design Aid), that is aimed at providing objective failure data on a candidate microkernel and also improving its error detection capabilities. The paper first presents the overall architecture of MAFALDA. Then, a case study carried out on an instance of the Chorus microkemel is used to illustrate the benefits that can be obtained with MAFALDA both from the dependability assessment and design-aid viewpoints. Implementation issues are also addressed that account for the specific API of the target microkemel. Some overall insights and lessons learned, gained during the various studies conducted on both Chorus and another target microkemel (LynxOS), are then depicted and discussed. Finally, we conclude the paper by summarizing the main features of the work presented and by identifying future research.

The separation of concerns provided by reflective architectures is of high interest for the development of dependable systems. Beyond this initial interest, the use of this technology remains questionable due to the lack of work reporting on validation aspects. This paper defines an incremental strategy for testing the cornerstone of the reflective architectures that we consider, i.e. the MetaObject protocol (MOP). The approach is aimed at reducing the testing effort by promoting a gradual increment of the observability and controllability of the MOP under test. This strategy enables reflective mechanisms that have already been tested to be re-used for verifying the remaining ones. A different testing level is associated with each reflective mechanism defined in the MOP. For each testing level, we characterize the test objectives and the required test environment. The feasibility of the approach is exemplified on a real MOP implemented using OpenC++ and extracted from the /spl Fscr//spl Rscr//spl Iscr//spl Escr//spl Nscr//spl Dscr//spl Sscr/ architecture, which is devoted to the implementation of CORBA dependable systems. First test experiments were very useful, since they revealed some implementation errors.

MAFALDA (Microkernel Assessment by Fault injection Analysis and Design Aid) is a generic tool providing quantitative information on COTS microkernels to support their integration into dependable systems. The main originality of MAFALDA relies on the features provided for both the analysis of the failure modes of the target microkernel and the design of error confinement wrappers. The paper illustrates: (i) how MAFALDA is organized and its user interface, and (ii) how it can be used to carry out fault injection campaigns. Finally, we present the experimental context of campaigns carried out on two commercial microkernels and draw the main lessons learnt.

This paper presents a method based on formal specifications for building robust real-time microkernels. Temporal logic is used to specify the functional and temporal properties of real-time kernels with respect to their main services (e.g., scheduling, time, synchronization, and clock interrupts). As an example of a synchronization mechanism, the specification of the Priority Ceiling Protocol is provided. The objective is to verify kernel properties at runtime in order to improve the internal kernel's detection mechanisms and complement their weaknesses. The core of this paper is a complete description of the temporal logic formulas corresponding to real-time kernel specifications. The formulas developed in this paper are the basis for the implementation of fault containment wrappers. The combination of COTS microkernels and wrappers leads to the notion of robust microkernels. The provided case study illustrates the approach on top of an instance of the Chorus microkernel.

The use of reflection is becoming popular today for the implementation of non-functional mechanisms such as fault tolerance. The main benefits of reflection are separation of concerns between the application and the mechanisms and transparency from the application programmer point of view. Unfortunately, metaobject protocols (MOPs) available today are not satisfactory with respect to necessary features needed for implementing fault tolerance mechanisms. Previously, we proposed a specialised MOP based on Corba, well adapted for such mechanisms (M.-O. Killijian and J.C. Fabre, 1998). We deliberately focus on the implementation of this metaobject protocol using compile-time reflection and its use for implementing distributed fault tolerance. We present the design and the implementation of a fault-tolerant Corba system using this metaobject together with some preliminary experimental results. From the lessons learnt from this work, we briefly address the benefits of reflection in other layers of a system for dependability issues.

This paper addresses the problem of using COTS microkernels for the design and implementation of dependable systems. Although not designed to deal with faulty situation, their analysis in the presence of faults is of high interest since all upper layers rely on their correct behavior. Such an information can be used to design upper layers accordingly. However, as shown in many works, the results obtained using fault injection on COTS executives raise unacceptable situations for critical applications. The definition of fault containment mechanisms is mandatory to deal with such situations and make simpler the design of upper layer software. The approach proposed in this paper relies on the modeling of microkernel functionalities. Modeling is indeed possible because these functions are often simple. The implementation of efficient fault containment wrappers proposed here is based on a novel approach taking advantage of the notion of reflective components.

The use of metalevel architectures for the implementation of fault-tolerant systems is today very appealing. Nevertheless, all such fault-tolerant systems have used a general-purpose metaobject protocol (MOP) or are based on restricted reflective features of some object-oriented language. According to our past experience, we define in this paper a suitable metaobject protocol, called FT-MOP for building fault-tolerant systems. We explain how to realize a specialized runtime MOP using compile-time reflection. This MOP is CORBA compliant: it enables the execution and the state evolution of CORBA objects to be controlled and enables the fault tolerance metalevel to be developed as CORBA software.

The FRIENDS system developed at LAAS-CNRS is a metalevel architecture providing libraries of metaobjects for fault tolerance, secure communication, and group-based distributed applications. The use of metaobjects provides a nice separation of concerns between mechanisms and applications. Metaobjects can be used transparently by applications and can be composed according to the needs of a given application, a given architecture, and its underlying properties. In FRIENDS, metaobjects are used recursively to add new properties to applications. They are designed using an object oriented design method and implemented on top of basic system services. This paper describes the FRIENDS software-based architecture, the object-oriented development of metaobjects, the experiments that we have done, and summarizes the advantages and drawbacks of a metaobject approach for building fault-tolerant systems.

An intrusion-tolerant distributed system is a system which is designed so that any intrusion into a part of the system will not endanger confidentiality, integrity and availability. This approach is suitable for distributed systems, because distribution enables isolation of elements so that an intrusion gives physical access to only a part of the system. In particular, the intrusion-tolerant authentication and authorization servers enable a consistent security policy to be implemented on a set of heterogeneous, untrusted sites, administered by untrusted (but nonconspiring) people. The authors describe how some functions of distributed systems can be designed to tolerate intrusions. A prototype of the persistent file server presented has been successfully developed and implemented as part of the Delta-4 project of the European ESPRIT program.>

The authors address the problem of validating the dependability of fault-tolerant computing systems, in particular, the validation of the fault-tolerance mechanisms. The proposed approach is based on the use of fault injection at the physical level on a hardware/software prototype of the system considered. The place of this approach in a validation-directed design process and with respect to related work on fault injection is clearly identified. The major requirements and problems related to the development and application of a validation methodology based on fault injection are presented and discussed. Emphasis is put on the definition, analysis, and use of the experimental dependability measures that can be obtained. The proposed methodology has been implemented through the realization of a general pin-level fault injection tool (MESSALINE), and its usefulness is demonstrated by the application of MESSALINE to the experimental validation of two systems: a subsystem of a centralized computerized interlocking system for railway control applications and a distributed system corresponding to the current implementation of the dependable communication system of the ESPRIT Delta-4 Project.>