Testing Web services performance and their Fault Tolerance Mechanisms (FTMs) are crucial for the development of today's applications. Testing the performance and FTMs of composed service systems is hard to measure at design time because service instability is often caused by the nature of the network. Using a real internet environment for testing is difficult to set up and control. We have developed a fault injection toolkit that emulates a WAN within a LAN environment between composed service components and offers full control over the emulated environments in addition to the ability to inject network-related and application specific faults. The tool also generates background workloads on the tested system for producing more realistic results. We describe an experiment that has been carried out to test the impact of fault tolerance protocols deployed at a service client by using our fault injection toolkit.

The increasing number of devices that are invisibly embedded into our surrounding environment as well as the proliferation of wireless communication and sensing technologies are the basis for visions like ambient intelligence, ubiquitous and pervasive computing. PErvasive Computing in Embedded Systems (PECES) project is committed to developing the technological basis to enable the global cooperation of embedded devices residing in different smart spaces in a context-dependent, secure and trustworthy manner. The PECES project provides a comprehensive software layer that consists of a flexible context ontology, a middleware that is capable of dynamically forming execution environments that are secure and trustworthy as well as a set of tools to facilitate application development.

It is usually infeasible to test distributed systems and protocols for dependability, fault-tolerance and performance in the real operation environment. This is attributed to the cost of setting up the hosting distributed environment and the difficulty of controlling such a dynamic environment. The factors are too many and are very dynamic. This paper presents the wide area network emulator for CORBA applications: a general purpose software tool that emulates a distributed networked environment for CORBA middleware and applications and gives the user full control over the emulation environment in addition to the ability to inject network-related faults

Replicated processing with majority voting is a well-known method for achieving reliability and availability. Triple modular redundant (TMR) processing is the most commonly used version of that method. Replicated processing requires that the replicas reach agreement on the order in which input requests are to be processed. Almost all synchronous and deterministic ordering protocols published in the literature are time-based in the sense that they require replicas' clocks to be kept synchronized within some known bound. We present a protocol for TMR systems that is based on timeouts and does not require clocks to be kept in bounded synchronism. Our design efforts focus on keeping the ordering delays small, without an unnecessary increase in message overhead. Consequently, we are able to show that no symmetric protocol that works only with unsynchronized clocks can provide a smaller worst-case delay. We also demonstrate through analysis and experiments that our protocol is faster than a time-based one of identical message complexity in certain situations which can prevail in many application settings.

Many fault-tolerant group communication middleware systems have been implemented assuming crash failure semantics. While this assumption is not unreasonable, it becomes hard to justify when applications are required to meet high reliability requirements and are built using commercial off the shelf (COTS) components. This paper presents a structured approach to extend a crash-tolerant middleware system into an authenticated Byzantine tolerant one with small modifications to the original system. The proposed approach is based on state machine replication (SMR) and is motivated by the composability features of standard distributed object technologies such as CORBA. SMR is used to assure signal-onfailure (fail-signal) semantics at a level where existing crash-tolerant services can be seamlessly deployed. The resulting system can provide total ordering that has no liveness requirement for termination. We demonstrate the effectiveness of our approach by porting a crash-tolerant CORBA group communication service – the NewTOP system. We also measure the performance of the resulting system.

Bloom filters make use of a ‘probabilistic’ hash-coding method to reduce the amount of space required to store a hash set. A Bloom filter offers a trade-off between its size and the probability that the filter returns the wrong result. It does this without storing the entire set, at the cost of occasionally incorrectly answering yes to the question ‘is $x$ a member of $s$?’. How Bloom filters can be used to speed up the name to location resolution process in large-scale distributed systems is discussed. The approach presented offers trade-offs between performance (the time taken to resolve an object's name to its location) and resource utilization (the amount of physical memory to store location information and the number of messages exchanged to obtain the object's address).

This paper uses fault injection to compare the ability of two fault-tolerant software architectures to protect an application from faults. These two architectures are Voltan, which uses process duplication, and Chameleon ARMORs, which use self-checking. The target application is a Dynamic Host Configuration Protocol (DHCP) server, a widely used application for managing IP addresses. NFTAPE, a software-based fault injection environment, is used to inject three classes of faults, namely random memory bit-flip, control-flow and high-level target specific faults, into each software architecture and into baseline Solaris and Linux versions.

A fail-silent node is a self-checking node that either functions correctly or stops functioning after an internal failure is detected. Such a node can be constructed from a number of conventional processors. In a software-implemented fail-silent node, the nonfaulty processors of the node need to execute message order and comparison protocols to "keep in step" and check each other, respectively. In this paper, the design and implementation of efficient protocols for a two processor fail-silent node are described in detail. The performance figures obtained indicate that in a wide class of applications requiring a high degree of fault tolerance, software-implemented fail-silent nodes constructed simply by utilizing standard "off-the-shelf" components are an attractive alternative to their hardware-implemented counterparts that do require special-purpose hardware components, such as fault-tolerant clocks, comparator, and bus interface circuits.

Replicated processing with majority voting is a well known method for achieving fault tolerance. Triple Modular Redundant (TMR) processing is the most commonly used version of that method. Replicated processing requires that the replicas reach agreement on the order in which messages are to be processed. Synchronous and deterministic ordering protocols published in the literature require that the replicas maintain an abstraction of clocks that are kept in known and bounded synchronism. We present a protocol for TMR systems that does not require this abstraction of synchronised clocks. We analyse the protocol performance and show that this protocol in practice can be at least as fast as any synchronised clock based ordering protocol. We also derive a faster protocol that has an improved performance in the absence of processor failures. We then build a TMR node and measure its performance to illustrate that the protocols developed here provide faster ordering and are easier to implement.

A VOLTAN node is composed of a number of conventional processors on which application-level processes are replicated to achieve fault-tolerance. The architecture of a family of such nodes with differing functionalities is presented. These include failure-masking, fail-signal, and fail-silent nodes. The software architectures of a three-processor failure-masking and a two-processor fail-silent node are discussed in detail. The suitability of VOLTAN nodes as building blocks of reliable distributed systems is also discussed.>