The maturity of hardware virtualization has motivated Communication Service Providers (CSPs) to apply thisparadigm to network services. Virtual Network Functions (VNFs)result from this trend and raise new dependability challengesrelated to network softwarisation that are still not thoroughlyexplored. This paper describes a new approach to detect ServiceLevel Agreements (SLAs) violations and preliminary symptomsof SLAs violations. In particular, one other major objectiveof our approach is to help CSP administrators to identify theanomalous VM at the origin of the detected SLA violation, whichshould enable them to proactively plan for appropriate recoverystrategies. To this end, we make use of virtual machine (VM)monitoring data and perform both a per-VM and an ensembleanalysis. Our approach includes a supervised machine learningalgorithm as well as fault injection tools. The experimental testbedconsists of a virtual IP Multimedia Subsystem developed by theClearwater project. Experimental results show that our approachcan achieve high precision and recall, and low false alarm rateand can pinpoint the root anomalous VNF VM causing SLAviolations. It can also detect preliminary symptoms of highworkloads triggering SLA violations.

Automotive embedded systems are becoming increasingly complex. Therefore verification activities are paramount to ensure safety. ISO 26262 is the first standard specifically dedicated to automotive safety systems. This standard requires introducing fault injection (FI) from the very early phases of the development process. Our work aims at developing an approach that will help integrate FI in the whole development process in a continuous way, from system requirements to the verification and validation phase. In this paper, we concentrate on exploring the benefits of safety analyses for experimental validation of the system. We propose an analogy between FI during the pre-implementation phase with safety analyses that are of common use during system design. We finally illustrate this approach on a case study from the automotive domain.

Scale-out storage systems (SoSS) have become increasingly important for meeting availability requirements of web services in cloud platforms. To enhance data availability, SoSS rely on a variety of built-in fault-tolerant mechanisms, including replication, redundant network topologies, advanced request scheduling, and other failover techniques. However, performance issues in cloud services still remain one of the main causes of discontentment among their tenants. In this paper, we propose an anomaly detection approach for SoSS that predicts cloud anomalies caused by memory and network faults. To evaluate our prediction model, we built a testbed simulating a virtual data center using VMware. Experimental results confirm that the injected faults are likely to undermine the data availability in SoSS. They suggest that although unsupervised learning has been the most common method for anomaly detection, a supervisedbased implementation of the same model reduces the false positive rate by roughly 10%. Our analysis also points out that probing SoSS-specific monitoring data at the VM-level contributes to improve the anomaly prediction efficiency.

This paper addresses an aircraft mission operational reliability as resulting from component failures, environment changes, and maintenance facilities offered at the various stops involved in the mission. We will show how the on-line assessment of operational reliability will help adjust an aircraft mission, in case of major changes to equipment availability during the mission. The assessment is made possible thanks to the building and validation of a generic dependability model that is easily i) processed for the assignment of an initial mission, and ii) updated during mission accomplishment, following the occurrence of some specific major events. The generic model can be built as early as the design phase, by engineers who are specialist in dependability assessment, based on stochastic processes. Model update and processing, during aircraft operation, can be achieved by operators who are not necessarily familiar with stochastic processes in the way that they are being applied in this research. We will present examples of results that show the valuable role of operational dependability re-assessment during aircraft mission.

Modern computer systems are increasing in complexity, spread, and scale in order to meet the diverse and sophisticated needs of the users. In the development of these systems, we inevitably use legacy codes and off-the shelf software as black box software in order to shorten the development time and lower the development cost. These systems are often connected via a network to utilize services provided by other systems, whereas services and network performance may change while in operation. We often need to change the specification and implementation of a system due to the changes of environments and users' requirements. In addition, threats caused by viruses and unauthorized accesses have to be properly removed. Therefore, modern computer systems inherently involve incompleteness of specifications and implementations and uncertainty of environments and requirements.

The four papers in this special section present new findings on different aspects of software dependability.

This paper addresses safety modeling and evaluation of automated highway systems, based on the use of platoons of vehicles driven by automated agents. We analyze the impact on safety of the strategy used to coordinate the vehicles' operations, inside each platoon and between platoons, when vehicles enter or exit the highway, or when maneuvers are carried out to recover from failures affecting the vehicles or their communication. To cope with the complexity of the studied system, a compositional approach based on stochastic activity networks is developed. Replicated submodels associated with each vehicle, describing the corresponding failure modes and recovery maneuvers and their severity, are composed with submodels characterizing the configuration of the platoons and their dynamic evolution. Numerical results are presented to highlight the impact of the coordination strategy and other dependability related parameters.

Recent experience and feedback from panels indicates that what the audience likes best is the chance to ask questions, particularly regarding things that might solve problems on their development projects or in their research studies. This panel has been held at every ISSRE since 1997, and has been highly popular. Originally the panel was chaired by John Musa, but Mike Hinchey took over that role last year. The panel has no presentations, only questions. The questions can be on anything, ranging from theory to details of application. The panelists have been selected to make available wide and extensive experience in the field.

In this paper, we introduce a simple analytical modeling approach for computing service unavailability due to long response time, for infinite and finite single-server systems as well as for multi-server systems. Closed-form equations of system unavailability based on the conditional response time distributions are derived and sensitivity analyses are carried out to analyze the impact of long response time on service unavailability. The evaluation provides practical quantitative results that can help distributed system developers in design decisions

The importance of benchmarking is increasing as every aspect of human life is relying on correct operation of computing systems. Although considerable efforts have been made, presently there are no widely accepted dependability benchmarks. The panelists, in the order shown above, address benchmarking of computer hardware, operating systems, applications, and systems.

This paper presents a dependability benchmark for general-purpose operating systems and its application to six versions of Windows operating system and four versions of Linux operating system. The benchmark measures are: operating system robustness (as regards possible erroneous inputs provided by the application software to the operating system via the application programming interface), reaction and restart times in the presence of faults. The workload is PostMark, a file system performance benchmark for operating systems.

Summary form only given. The explosive growth of Internet and wireless infrastructure allows software development collaboratively from multiple locations. Technology and standards have evolved to the point where it is feasible to perform full life cycle software development through multiple site cooperation. In fact, most large software corporations have distributed or even global software development, an example of which is the current trend of outsourcing to India and China. Much research in the area of CSCW concentrated on the area of improving efficiency of groupware. Topics include group awareness, multi-user interfaces, concurrency control, communication and coordination within the group, shared information space and the support of a heterogenous, open environment which integrates existing single-user applications, and etc. The focus is on application of CSCW technology to software engineering regarding collaborative software development, including design, workflow, testbeds, and etc. In particular, we would like to discuss one fundamental question, i.e. whether CSCW software development is more efficient than single-location software development. The topics of our discussion include, but are not limited to the following: 1. What is the impact of CSCW on software reliability, as compared with single-site software development? 2. What is the impact of CSCW on software productivity as compared with single site software development? 3. What are the benefits of CSCW for software engineering? 4. What are the new research topics in CSCW for software engineering? 5. How can CSCW be used in software design, development and testing?.

The aim of this paper is to compare the dependability of three operating systems (Windows NT4, Windows 2000 and Windows XP) with respect to erroneous behavior of the application layer. The results show a similar behavior of the three OSs with respect to robustness and a noticeable difference in OS reaction and restart times. They also show that the application state (mainly the hang and abort states) significantly impacts the restart time for the three OSs.

We propose a dependability benchmark suitable for a general purpose operating system (OS). The specifications of the benchmark components are presented and illustrated on a benchmark prototype dedicated to Windows 2000. The important novelty, as regards OS dependability benchmarking, is threefold. First, it lies on a comprehensive and structured set of measures: outcomes are considered both at the OS level and at the application level. Second, these measures include not only robustness measures (e.g., the distribution among the observed outcomes for the OS and the application: error codes, exceptions, workload correct or erroneous completion, OS and application hang), but also the related temporal measures in the presence of faults (e.g., system call and workload execution times, as well as operating system restart time). Finally, we are considering a realistic workload (namely, TPC-C client), instead of a synthetic workload.

International audience

Abstract This paper defines a multi‐level modeling approach for availability assessment of e‐business applications, based on two main steps: (1) hierarchical description of the system and its interactions with the users, from the functional and structural points of view, and (2) hierarchical construction and solution of the availability models based on information from the first step. Four modeling abstraction levels are considered. The highest level (user level) describes the availability of the e‐business application as perceived by the users. Intermediate levels describe the availability of functions and services provided to the users. The lowest level describes the availability of the component systems on which functions and services are implemented. The availability measures of a given level are computed based on the measures provided by the immediately lower level. Copyright © 2003 John Wiley & Sons, Ltd.

The aim of our work is to provide a modeling framework for evaluating performability measures of Multipurpose, Multiprocessor Systems (MMSs). The originality of our approach is in the explicit separation between the architectural and environmental concerns of a system. The overall dependability model, based on stochastic reward nets, is composed of 1) an architectural model describing the behavior of system hardware and software components, 2) a service-level model, and 3) a maintenance policy model. The two latter models are related to the system utilization environment. The results can be used for supporting the manufacturer design choices as well as the potential end-user configuration selection. We illustrate the approach on a particular family of MMSs under investigation by a system manufacturer for Internet and e-commerce applications. As the systems are scalable, we consider two architectures: a reference one composed of 16 processors and an extended one with 20 processors. Then, we use the obtained results to evaluate the performability of a clustered system composed of four reference systems. We evaluate comprehensive measures defined with respect to the end-user service requirements and specific measures in relation to the distributed shared memory paradigm.

This paper presents a stepwise approach for dependability modeling, based on Generalized Stochastic Petri Nets (GSPNs). The first-step model called functional-level model, can be built as early as system functional specifications and then completed by the structural model as soon as the system architecture is known, even at a very high level. The latter can be refitted according to three different aspects: component decomposition, state and event fine-tuning and distribution adjustment to take into account increasing event rates. We define specific rules to make the successive transformations as easy and systematic as possible. This approach allows the various dependencies to be taken into account at the right level of abstraction: functional dependency, structural dependency and those induced by non-exponential distributions. A part of the approach is applied to an instrumentation and control system (I&C) in power plants.

The main goal of the experimental stud), reported in this paper is to investigate to what extent distinct fault injection techniques lead to similar consequences (errors and failures). The target system we are using to carry out our investigation is the Linux kernel as it provides a representative operating system. It is featuring full controllability and observability thanks to its open source status. Three types of software-implemented fault injection techniques are considered, namely: i) provision of invalid values to the parameters of the kernel calls, ii) corruption of the parameters of the kernel calls, and iii) corruption of the input parameters of the internal functions of the kernel. The workload being used for the experiments is tailored to activate selectively each functional component. The observations encompass typical kernel failure modes (e.g., exceptions and kernel hangs) as well as a detailed analysis of the reported error codes.

The application of fault injection in the context of dependability benchmarking is far from being straightforward. One decisive issue to be addressed is to what extent injected faults are representative of actual faults. This paper proposes an approach to analyze the effects of real and injected faults.

This paper defines a multi-level modeling framework for E-business system availability evaluation. It is based on two main steps: (1) hierarchical description of the system and its interactions with the users, from the functional and structural points of view; and (2) hierarchical construction and solution of the system availability models based on information from the first step.

This paper presents a framework for modeling the dependability of hardware and software fault-tolerant systems, taking into account explicitly the dependence among the components. These dependencies can result from: (a) functional or structural interactions between the components or (b) interactions due to global system reconfiguration and maintenance strategies. Modeling is based on GSPN (generalized stochastic Petri net). The modeling approach is modular: the behavior of each component and each interaction is represented by its own GSPN, while the system model is obtained by composition of these GSPN. Composition rules are defined and formalized through clear identification of the interfaces between the component and interaction nets. In addition to modularity, the formalism brings flexibility and re-usability, thereby allowing easy sensitivity analysis with respect to the assumptions that could be made about the behavior of the components and the resulting interactions. This approach has been successfully applied to select new architectures for the French Air Traffic Control system, based among other things, on availability evaluation. This paper illustrates it on a simple representative example, including all the types of the identified dependencies: the duplex system. Modeling of this system showed the strong dependence between components.

This paper analyzes data related to working hours that have been recorded over seven years, during the development of a real-life software system. The software under consideration is composed of two diverse (dissimilar) units, called variants. The second variant is used for self-checking. The results of the two variants are compared: in case of agreement, the outputs of the principal variant are transmitted and in case of discrepancy an error is reported. For each development phase, we evaluate the cost overhead induced by the development of the second variant with respect to the cost of the principal variant, considered as a reference. The results show that the cost overhead varies from 25% to 134% according to the development phase.

The aim of our work is to provide a quantified means of helping in the definition of a new architecture for CAUTRA, a subset of the French Air Traffic Control system. In this paper, we define a set of alternative architectures, give some elements for constructing their dependability models, and compare their availability. Modeling is carried out following a modular and systematic approach, based on the derivation of block models at a high level of abstraction. In a second step, the blocks are replaced by their equivalent Generalized Stochastic Petri Nets to build up the detailed model of the architecture. The evaluations performed permit identification of a subset of architectures whose availability meets the dependability requirements and also identification of the best architecture among this subset.

Analyzes data collected on a commercial telecommunications system and summarizes some of the lessons learned from this study. The data correspond to failure and fault information recorded during system validation and operation: 3,063 trouble reports, corresponding to a five-year period during which five versions of the system have been developed and more than 100 systems have been introduced in the field. The failure information includes software failures as well as hardware failures due to design faults.

This paper addresses the problem of reliability growth characterization and analysis. It is intended to show how reliability trend analyses can help the project manager in controlling the progress of the development activities and in appreciating the efficiency of the test programs. Reliability trend change may result from various reasons, some of them are desirable and expected (such as reliability growth due to fault removal) and some of them are undesirable (such as slowing down Of the testing effectiveness). Identification in time of the latter allows the project manager to take the appropriate decisions very quickly in order to avoid problems which may manifest later. The notions of reliability growth over a given interval and local reliability trend change are introduced through the subadditive property, allowing: better definition and understanding of the reliability growth phenomena; the already existing trend tests are then revisited using these concepts. Emphasis is put on the way trend tests can be used to help the management of the testing and validation process and on practical results that can be derived from their use; it is shown that, for several circumstances, trend analyses give information of prime importance to the developer.>

This paper describes the different steps in software failure data processing in order to monitor the software development and to quantify the operational reliability. Processing consists in (a) filtering the raw data in order to keep only those corresponding to software failures without duplicate, (b) partitioning of data into sub-sets according for instance to failure severity or fault location, (c) performing descriptive analyses, (d) analyzing the trend and (e) when possible and needed, applying reliability growth models. These steps are part of an overall method experienced at LAAS on several real-life software systems. The goals of the paper are twofold: first, present the method and, second, show its benefits through its application to data collected on a switching system.

This paper is devoted to the definition of a reliability growth model-referred to as the discrete time hyperexponential model. This model is aimed at modeling software reliability with respect to the number of executions performed. It is well-suited to some kinds of systems such as transaction processing systems, single mission systems, etc. for which discrete data are collected; in addition, it facilities the modeling of the impact on dependability measures of some software environment characteristics, such as the input probability distribution.>

The problem of modeling a system's reliability and availability with respect to the various classes of faults (physical and design, internal and external) which may affect the service delivered to its users is addressed. Hardware and software models are currently exceptions in spite of the user's requirements; these requirements are expressed in terms of failures independently of their sources, i.e., the various classes of faults. The causes of this situation are analyzed; it is shown that there is no theoretical impediment to deriving such models, and that the classical reliability theory can be generalized in order to cover both hardware and software viewpoints that are X-Ware.>

The authors report a study of the dependability of the various communication topologies that can be used to construct a Delta-4 system. Single and dual bus and ring configurations are possible (based on 802.4, 802.5, and FDDI standards); the authors give closed-form expressions for the reliability and availability of each topology when repair is taken into account. It is shown that the dimensioning parameter in the dependability of the communication system is the coverage of the self-checking mechanisms built into the network attachment controllers.>

An evaluation method which allows existing reliability growth models to provide better predictions of software behavior is presented. The method is primarily based on the analysis of the trend exhibited by the data collected on the program (which is determined by reliability growth tests). Reliability data are then partitioned according to the trend, and two types of reliability growth models can be applied: when the data exhibit reliability decrease followed by reliability growth, an S-shaped model can be applied, and in case of reliability growth, most of the other existing reliability growth models can be applied. The hyperexponential model is shown to allow prediction of the software residual failure rate in operation, and this failure rate is used as a qualification index for the software product. The method is illustrated through its application to the Brazilian electronic switching system TROPICO-R.>

An approach for the modeling and evaluation of reliability and availability of systems using the knowledge of the reliability growth of their components is presented. Detailed models of reliability and availability for single-component systems are derived under much weaker assumption than usually considered. These models, termed knowledge models, enable phenomena to be precisely characterized, and a number of properties to be deduced. Since the knowledge models are too complex to be applied in real life for performing predictions, simplified models for practical purposes (action models) are discussed. The hyperexponential model is presented and applied to field data of software and hardware failures. This model is shown to be comparable to other models as far as reliability of single-component systems is concerned: in addition, it enables estimating and predicting the reliability of multicomponent systems, as well as their availability. The transformation approach enables classical Markov models to be transformed into other Markov models which account for reliability growth. The application of the transformation to multicomponent systems is described.>

Dependability modeling and evaluation (encompassing reliability and safety issues) of the two major fault tolerance software approaches-recovery blocks (RBs) and N version programming (NVP)-are presented. The study is based on the detailed analysis of software fault-tolerance architectures able to tolerate a single fault (RB: two alternates and an acceptance test; NVP: three versions and a decider). For each approach a detailed model based on the software production process is established and then simplified by assuming that only a single fault type may manifest during execution of the fault-tolerant software and that no error compensation may take place within the software. The analytical results obtained make it possible to identify the improvement, compared to a non-fault-tolerant software, that could result from the use of RB (the acceptance test has to be more reliable from the alternates) and NVP (related faults among the versions and the decider have to be minimized) and to determine the most critical types of related faults. Nested RBs are studied, showing that the proposed analysis approach can be applied to such realistic software structures and that when an alternate is itself an RB, the results are analogous to the case of the addition of a third alternate. The reliability analysis shows that only a small improvement can be expected.>