Edge computing has received much attention from industry and academia during the past few years, owing to its inherent merits in improving latency and saving band-width. Unlike cloud computing which is supported by over two decades of research, edge computing is a relatively new field of study. Therefore, important challenges such as dependability and fault tolerance have not been fully explored. This poses potential risks for the rapidly growing adoption of the edge computing paradigm. This paper summarizes edge computing's dependability challenges, emerging use cases, related industry standards, and state-of-the-art fault tolerance research in this field. Furthermore, we present the most pertinent open research problems and opportunities in the field of fault-tolerant edge computing.

Recognition and perception based mobile applications, such as image recognition, are on the rise. These applications recognize the user's surroundings and augment it with information and/or media. These applications are latency-sensitive. They have a soft-realtime nature - late results are potentially meaningless. On the one hand, given the compute-intensive nature of the tasks performed by such applications, execution is typically offloaded to the cloud. On the other hand, offloading such applications to the cloud incurs network latency, which can increase the user-perceived latency. Consequently, edge computing has been proposed to let devices offload intensive tasks to edge servers instead of the cloud, to reduce latency. In this paper, we propose a different model for using edge servers. We propose to use the edge as a specialized cache for recognition applications and formulate the expected latency for such a cache. We show that using an edge server like a typical web cache, for recognition applications, can lead to higher latencies. We propose Cachier, a system that uses the caching model along with novel optimizations to minimize latency by adaptively balancing load between the edge and the cloud, by leveraging spatiotemporal locality of requests, using offline analysis of applications, and online estimates of network conditions. We evaluate Cachier for image-recognition applications and show that our techniques yield 3x speedup in responsiveness, and perform accurately over a range of operating conditions. To the best of our knowledge, this is the first work that models edge servers as caches for compute-intensive recognition applications, and Cachier is the first system that uses this model to minimize latency for these applications.

Recent advances in mobile device technology have triggered research on using their aggregate computational and/or storage resources to form edge-clouds. Whilst traditionally viewed as simple clients, smartphones and tablets today have hardware resources that allow more sophisticated software to be installed, and can be used as thick clients or even thin servers. Simultaneously, new standards and protocols, such as Wi-Fi Direct and Wi-Fi TDLS (Tunneled Direct Link Setup), have been established that allow mobile devices to talk directly with each other, as opposed to over the Internet or across Wi-Fi access points. This can, potentially, lead to ubiquitous, low-latency, device-to-device (D2D) communication. In this paper, we study whether D2D protocols can support mobile-edge clouds by benchmarking different protocols and configurations for a specific application. The results show that decentralized device-to-device techniques can be used to efficiently disseminate multi- media contents while diminishing contention in the wireless infrastructure, allowing for up to 65 % traffic reduction at the access points.

Control-Flow Integrity (CFI) is an important safety property of software, particularly in embedded and safety-critical systems, where CFI violations have led to patient deaths and can render cars remotely controllable by attackers. Previous techniques for CFI may reduce the robustness of embedded and safety-critical systems, as they handle CFI violations by stopping programs. In this work, we present PCFIRE, a preventative approach to CFI that prevents the root-causes of CFI violations to allow recovery, and enables programmers to specify robust recovery actions by providing CFI via source-code safety-checks. PCFIRE's CFI can be formally proved automatically, and supports realistic features of embedded software such as hardware and I/O access. We showcase PCFIRE by providing, and automatically proving, CFI for: benchmark programs, text utilities containing I/O, and embedded programs with sensor inputs and hardware outputs on the Raspberry Pi single-board computer.

Zephyr is a first-person wireless-performance data-collection approach to study the end-user's perspective of the wireless-network's performance using an OTT video-streaming service used in 35 mobile applications and 25 sports stadiums. We use Zephyr to provide insights into user behavior and user disengagement from production traces gathered over 2+ years. We identify the different types of failures that we've observed, and describe how frequently they occur. Finally, we correlate low-level Wi-Fi performance data with application-level failures and describe the trends that we observe. The data from Zephyr reveals that 50% of users disengage if video playback does not begin within 12 seconds, and also if video playback stalls, instream, for more than 6 seconds. The most dominant failure type for video playback is cancellation during the initial buffering phase, which accounts for 30% of all streams.

YinzCam is a cloud-hosted service that provides sports fans with real-time scores, news, photos, statistics, live radio, streaming video, etc., on their mobile devices. YinzCam’s infrastructure is currently hosted on Amazon Web Services (AWS) and supports over 30 million installs of the official mobile apps of 140+ NHL/NFL/NBA/NRL/NCAA sports teams and venues. YinzCam’s workload is necessarily multi-modal (e.g., pre-game, in-game, post-game, game-day, non-gameday), with normal game-time traffic being twenty-fold of that on non-game days. This paper describes the evolution of YinzCam’s production architecture and distributed infrastructure, from its beginnings in 2009, when it was used to support thousands of concurrent users, to today’s system that supports millions of concurrent users on any game day. We also discuss key new opportunities to improve the fan experience inside the stadium of the future, without impacting the available bandwidth, by crowd-sourcing the thousands of mobile devices that are in fans’ hands inside these venues. We present Krowd, a novel distributed key-value store for promoting efficient content sharing, discovery and retrieval across the mobile devices inside a stadium. We present CHIPS, a system that ensures that users’ privacy is maintained while their devices participate in the crowdsourced infrastructure.

The rapid growth in mobile devices will give rise to the trend of the leasing out of compute and data resources on mobile devices to third-parties for applications to be run on multiple mobile devices. However, these third-party applications running on leased mobile devices are typically written by unknown entities, and cannot be trusted by mobile device owners. Current mobile device platforms (e.g. Android) have permissions and access control systems designed for mobile apps that are written by reputable developers and vetted by authoritative app stores, and they are not suitable for untrusted apps. We propose STOVEPipe, an observable access control system for user data on mobile devices for untrusted third-party applications. STOVEPipe ensures that untrusted code is isolated and cannot directly access system data, and performs all data accesses on behalf of untrusted apps. This enables STOVEPipe to observe all data accessed by untrusted apps, implement content-based access control, perform accounting and auditing on accessed data easily, and perform privacy-preserving data transformations.

The massive growth in mobile devices is likely to give rise to the leasing out of compute and data resources on mobile devices to third-parties to enable applications to be run across multiple mobile devices. However, users who lease their mobile devices out need to run applications from unknown third-parties, and these untrusted applications may harm their devices or access unauthorized personal data. We propose STOVE, a data and execution model for structuring untrusted applications to be secure by construction, to achieve strict and verifiable execution isolation, and observable access control for data. STOVE uses formal logic to verify that untrusted code meets isolation properties which imply that hosts running the code cannot be harmed, and that untrusted code cannot directly access host data. STOVE performs all data accesses on behalf of untrusted code, allowing all access control decisions to be reliably performed in one place. Thus, users can run untrusted applications structured using the STOVE model on their systems, with strong guarantees, based on formal proofs, that these applications will not harm their system nor access unauthorized data.

The Android permissions system provides all-or-nothing access to users' photos stored on smartphones, and the permissions which control access to stored photos can be confusing to the average user. Our analysis found that 73% of the top 250 free apps on the Google Play store have permissions that may not reflect their ability to access stored photos. We propose CHIPS, a unique content-based fine-grained run-time access control system for stored photos for Android which requires minimal user assistance, runs entirely locally, and provides low-level enforcement. CHIPS can recognize faces with minimal user training to deny apps access to photos with known faces. CHIPS's privacy identification has low overheads as privacy checks are cached, and is accurate, with false-positive and false-negative rates of less than 8%.

Chronics are recurrent problems that often fly under the radar of operations teams because they do not affect enough users or service invocations to set off alarm thresholds. In contrast with major outages that are rare, often have a single cause, and as a result are relatively easy to detect and diagnose quickly, chronic problems are elusive because they are often triggered by complex conditions, persist in a system for days or weeks, and coexist with other problems active at the same time. In this paper, we present Draco, a scalable engine to diagnose chronics that addresses these issues by using a “top-down” approach that starts by heuristically identifying user interactions that are likely to have failed, e.g., dropped calls, and drills down to identify groups of properties that best explain the difference between failed and successful interactions by using a scalable Bayesian learner. We have deployed Draco in production for the VoIP operations of a major ISP. In addition to providing examples of chronics that Draco has helped identify, we show via a comprehensive evaluation on production data that Draco provided 97% coverage, had fewer than 4% false positives, and outperformed state-of-the-art diagnostic techniques by up to 56% for complex chronics.

The ability to identify semantically-related functions, in large collections of binary executables, is important for malware detection. Intuitively, two pieces of code are similar if they have the same effect on a machine's state. Current state-of-the-art tools employ a variety of pair wise comparisons (e.g., template matching using SMT solvers, Value-Set analysis at critical program points, API call matching, etc.) However, these methods are unshakable for clustering large datasets, of size N, since they require O(N2) comparisons. In this paper, we present an alternative approach based upon "hashing". We propose a scheme that captures the semantics of functions as semantic hashes. Our approach treats a function as a set of features, each of which represent the input-output behavior of a basic block. Using a form of locality-sensitive hashing known as Min Hashing, functions with many common features can be quickly identified, and the complexity of clustering is reduced to O(N). Experiments on functions extracted from the CERT malware catalog indicate that we are able to cluster closely related code with a low false positive rate.

MapReduce is a programming paradigm for parallel processing that is increasingly being used for data-intensive applications in cloud computing environments. An understanding of the characteristics of workloads running in MapReduce environments benefits both the service providers in the cloud and users: the service provider can use this knowledge to make better scheduling decisions, while the user can learn what aspects of their jobs impact performance. This paper analyzes 10-months of MapReduce logs from the M45 supercomputing cluster which Yahoo! made freely available to select universities for academic research. We characterize resource utilization patterns, job patterns, and sources of failures. We use an instance-based learning technique that exploits temporal locality to predict job completion times from historical data and identify potential performance problems in our dataset.

Standardized software architectures, such as AUTomotive Open System ARchitecture (AUTOSAR), are being pursued within the automotive industry in order to reduce the cost of developing new vehicle features. Many of these features will need to be highly dependable. Fault injection plays an important role during the dependability analysis of such software. This work evaluates the feasibility of leveraging the CANoe simulation environment to develop software-based methods for injecting faults into AUTOSAR applications. We describe a proof-of-concept fault-injection framework with example fault-injection scenarios, as well as implementation issues faced and addressed, lessons learned, and the suitability of using CANoe as a fault-injection environment.

The distributed nature and large scale of MapReduce programs and systems poses two challenges in using existing profiling and debugging tools to understand MapReduce programs. Existing tools produce too much information because of the large scale of MapReduce programs, and they do not expose program behaviors in terms of Maps and Reduces. We have developed a novel non-intrusive log-analysis technique which extracts state-machine views of the control- and data-flows in MapReduce behavior from the native logs of Hadoop MapReduce systems, and it synthesizes these views to create a unified, causal view of MapReduce program behavior. This technique enables us to visualize MapReduce programs in terms of MapReduce-specific behaviors, aiding operators in reasoning about and debugging performance problems in MapReduce systems. We validate our technique and visualizations using a realworld workload, showing how to understand the structure and performance behavior of MapReduce jobs, and diagnose injected performance problems reproduced from real-world problems.

Black-box instrumentation can support problem diagnosis in distributed systems without the need to modify the application code or to understand its semantics. We explore a novel, low-overhead black-box instrumentation source - CPU/hardware performance counters - for problem diagnosis. Based on our hypothesis that performance problems manifest as observable, anomalous changes in CPU performance counter-values collected across the nodes of a distributed system, we develop a diagnosis approach that is able to detect and localize performance problems injected into RUBiS, a three-tier e-commerce system.

We present Kahuna, an approach that aims to diagnose performance problems in MapReduce systems. Central to Kahuna's approach is our insight on peer-similarity, that nodes behave alike in the absence of performance problems, and that a node that behaves differently is the likely culprit of a performance problem. We present applications of Kahuna's insight in techniques and their algorithms to statistically compare black-box (OS-level performance metrics) and white-box (Hadoop-log statistics) data across the different nodes of a MapReduce cluster, in order to identify the faulty node(s). We also present empirical evidence of our peer-similarity observations from the 4000-processor Yahoo! M45 Hadoop cluster. In addition, we demonstrate Kahuna's effectiveness through experimental evaluation of two algorithms for a number of reported performance problems, on four different workloads in a 100-node Hadoop cluster running on Amazon's EC2 infrastructure.

Online software upgrades are often plagued by runtime behaviors that are poorly understood and difficult to ascertain. For example, the interactions among multiple versions of the software expose the system to race conditions that can introduce latent errors or data corruption. Moreover, industry trends suggest that online upgrades are currently needed in large-scale enterprise systems, which often span multiple administrative domains (e.g., Web 2.0 applications that rely on AJAX client-side code or systems that lease cloud-computing resources). In such systems, the enterprise does not control all the tiers of the system and cannot coordinate the upgrade process, making existing techniques inadequate to prevent mixed-version races. In this paper, we present an analytical framework for impact assessment, which allows system administrators to directly compare the risk of following an online-upgrade plan with the risk of delaying or canceling the upgrade. We also describe an executable model that implements our formal impact assessment and enables a systematic approach for deciding whether an online upgrade is appropriate. Our model provides a method of last resort for avoiding undesirable program behaviors, in situations where mixed-version races cannot be avoided through other technical means.

We address the problem of providing transparent, lightweight, fault-tolerance mechanisms for generic peer-to-peer middleware systems. The main idea is to use the peer-to-peer overlay to provide for fault-tolerance rather than support it higher up in the middleware architecture, e.g. in the form of services. To evaluate our approach we have implemented a fault-tolerant middleware prototype that uses a hierarchical peer-to-peer overlay in which the leaf peers connect to sensors that provide data streams. Clients connect to the root of the overlay and request streams that are routed upwards through intermediate peers in the overlay up to the client. We report encouraging preliminary results for latency, jitter and resource consumption for both the non-faulty and faulty cases.

This paper describes our experiences with developing cost-effective assistive technologies for the visually impaired, with a focus on using commercial off-the-shelf technologies as much as possible. Trinetra involves three specific technologies--the grocery shopping assistant, the currency identifier and the transportation assistant--all supported on standard mobile phones with text-to-speech, commonly used by the visually impaired.

Replicated file-systems can experience degraded performance that might not be adequately handled by the underlying fault-tolerant protocols. We describe the design and implementation of Gumshoe, a system that aims to diagnose performance problems in replicated file-systems. Gumshoe periodically gathers OS and protocol metrics and then analyzes these metrics to automatically localize the performance problem to the culprit node(s). We describe our results and experiences with problem diagnosis in two replicated file-systems (replicated-CoreFS and BFS) using two file-system benchmarks (Postmark and IOzone).

Faults in distributed systems can result in errors that manifest in several ways, potentially even in parts of the system that are not collocated with the root cause. These manifestations often appear as deviations (or "errors") in performance metrics. By transparently gathering, and then identifying escalating anomalous behavior in, various node-level and system-level performance metrics, the Tiresias system makes black-box failure-prediction possible. Through the trend analysis of performance metrics, Tiresias provides a window of opportunity (look-ahead time) for system recovery prior to impending crash failures. We empirically validate the heuristic rules of the Tiresias system by analyzing fault-free and faulty performance data from a replicated middleware-based system.

We present Castor, a secure code-update protocol for sensor networks that exploits symmetric cryptoystems. Through a synergistic combination of a one-way hash-chain, two one-way key-chains with the delayed disclosure of symmetric keys, and multiple message authentication codes (MACs), Castor enables untrusted sensor nodes to verify an update's authenticity and guarantees that no correct node will ever install or forward a compromised part of a code-update image. We describe an implementation of Castor that hardens the TinyOS-based update protocol, Deluge, against node compromise. We experimentally compare Castor's computational and communication costs with those of Deluge and with those of a contemporary secure update protocol, Sluice, that uses asymmetric cryptosystems (digital signatures) instead. Our results demonstrate that Castor incurs reasonable overheads as compared to Deluge, and lower resource usage as well as lower end-to-end update latency as compared to Sluice.

Existing network reprogramming protocols target the efficient, reliable, multi-hop dissemination of application updates in sensor networks, but assume correct or fail-stop behavior from participating sensors. Compromised nodes can subvert such protocols to result in the propagation and remote installation of malicious code. Sluice aims for the progressive, resource-sensitive verification of updates in sensor networks to ensure that malicious updates are not disseminated or installed, while trusted updates continue to be efficiently disseminated. Our verification mechanism provides authenticity and integrity through a hash-chain construction that amortizes the cost of a single digital signature over an entire update. We integrate Sluice with an existing network reprogramming protocol and empirically evaluate its effectiveness both in a real sensor testbed and through simulation.

Existing code update protocols target efficiency and assume correct behavior from participating sensor nodes. This work aims for the progressive, resource sensitive verification of code updates in sensor networks to ensure that unauthorized updates from malicious nodes are not propagated, while correct updates continue to be efficiently disseminated.

Distributed applications composed of collections of Web services may call for diverse levels of reliability in different parts of the system. Byzantine fault tolerance (BFT) is a general strategy that has recently been shown to be practical for the development of certain classes of survivable, client-server, distributed applications; however, little research has been done on incorporating it into selective parts of multi-tier, distributed applications like Web services that have heterogeneous reliability requirements. To understand the impacts of combining BFT and Web services, we have created Thema, a new BFT middleware system that extends the BFT and Web services technologies to provide a structured way to build Byzantine-fault-tolerant, survivable Web services that application developers can use like other Web services. From a reliability perspective, our enhancements are also novel in that they allow Byzantine-fault-tolerant services: (1) to support the multi-tiered requirements of Web services, and (2) to provide standardized Web services support for their own clients (through WSDL interfaces and SOAP communication). In this paper we study key architectural implications of combining BFT with Web services and provide a performance evaluation of Thema using the TPC-W benchmark.

Abstract The OMG's Real‐Time CORBA (RT‐CORBA) and Fault‐Tolerant CORBA (FT‐CORBA) specifications make it possible for today's CORBA implementations to exhibit either real‐time or fault tolerance in isolation. While real‐time requiresa prioriknowledge of the system's temporal operation, fault tolerance necessarily deals with faults that occur unexpectedly, and with possibly unpredictable fault recovery times. The MEAD (Middleware for Embedded Adaptive Dependability) system attempts to identify and to reconcile the conflicts between real‐time and fault tolerance, in a resource‐aware manner, for distributed CORBA applications. MEAD supports transparent yet tunable fault tolerance in real‐time, proactive dependability, resource‐aware system adaptation to crash, communication and timing faults with bounded fault detection and fault recovery. Copyright © 2005 John Wiley & Sons, Ltd.

Embedded systems encompass a wide range of applications, technologies, and disciplines, necessitating a broad approach to education. We describe embedded system coursework during the first 4 years of university education (the U.S. undergraduate level). Embedded application curriculum areas include: small and single-microcontroller applications, control systems, distributed embedded control, system-on-chip, networking, embedded PCs, critical systems, robotics, computer peripherals, wireless data systems, signal processing, and command and control. Additional cross-cutting skills that are important to embedded system designers include: security, dependability, energy-aware computing, software/systems engineering, real-time computing, and human--computer interaction. We describe lessons learned from teaching courses in many of these areas, as well as general skills taught and approaches used, including a heavy emphasis on course projects to teach system skills.

Unanticipated runtime events, such as faults, can lead to missed deadlines in real-time systems. While it is not always possible to know when a fault will occur, we can sometimes exploit pre-fault "symptoms" to initiate proactive (rather than reactive) fault-recovery. In this paper, we describe the design and implementation of a proactive recovery strategy for distributed CORBA applications in the presence of resource-exhaustion faults. We analyze the effect of different proactive recovery schemes on client/server response times, and we demonstrate a significant reduction, both in jitter and in the number of client-side failures.

Fault-tolerant replicated applications are typically assumed to be deterministic, in order to ensure reproducible, consistent behavior and state across a distributed system. Real applications often contain nondeterministic features that cannot be eliminated. Through the novel application of program analysis to distributed CORBA applications, we decompose an application into its constituent structures, and discover the kinds of nondeterminism present within the application. We target the instances of nondeterminism that can be compensated for automatically, and highlight to the application programmer those instances of nondeterminism that need to be manually rectified. We demonstrate our approach by compensating for specific forms of nondeterminism and by quantifying the associated performance overheads. The resulting code growth is typically limited to one extra line for every instance of nondeterminism, and the runtime overhead is minimal, compared to a fault-tolerant application with no compensation for nondeterminism.

It has been almost a decade since the earliest reliable CORBA implementation and, despite the adoption of the fault-tolerant CORBA (FT-CORBA) standard by the Object Management Group, CORBA is still not considered the preferred platform for building dependable distributed applications. Among the obstacles to FT-CORBA's widespread deployment are the complexity of the new standard, the lack of understanding in implementing and deploying reliable CORBA applications, and the fact that current FT-CORBA do not lend themselves readily to complex, real-world applications. We candidly share our independent experiences as developers of two distinct reliable CORBA infrastructures (OGS and Eternal) and as contributors to the FT-CORBA standardization process. Our objective is to reveal the intricacies, challenges, and strategies in developing fault-tolerant CORBA systems, including our own. Starting with an overview of the new FT-CORBA standard, we discuss its limitations, along with techniques for best exploiting it. We reflect on the difficulties that we have encountered in building dependable CORBA systems, the solutions that we developed to address these challenges, and the lessons that we learned. Finally, we highlight some of the open issues, such as nondeterminism and partitioning, that remain to be resolved.

The Eternal system pioneered the interception approach to providing transparent fault tolerance for CORBA, which allows it to make a CORBA application reliable with little or no modification to the application or the ORB. The design and implementation of the Eternal system has influenced industrial practices by providing the basis for the specifications of the fault-tolerant CORBA standard that the Object Management Group adopted. We discuss our experience in developing the Eternal system, with particular emphasis on the challenges that we encountered and the lessons that we learned.

Abstract Enterprises are increasingly involved in worldwide round‐the‐clock e‐commerce and e‐business, which requires them to be operational 24 hours per day, 7 days per week. With outages leading to loss of revenue, reputation and customers, fault tolerance becomes increasingly important. By mixing the fault tolerance logic into the application logic, existing fault tolerance practices render applications more complex, more prone to errors, and more difficult to maintain and build. The Eternal system is a component‐based middleware framework that provides transparent fault tolerance for enterprise applications, and that ensures continuous$24\times7$ operation without requiring special skills of the application programmers. The Eternal system implements the new Fault‐Tolerant CORBA standard. Copyright © 2002 John Wiley & Sons, Ltd.

The Eternal system provides transparent fault tolerance for CORBA applications, without requiring the modification of either the application or the ORB. Eternal replicates the application objects, and ensures strong replica consistency by employing reliable totally-ordered multicast messages for conveying the IIOP messages of the application. To maintain replica consistency even as replicas fail and are recovered, Eternal ensures the retrieval, assignment and transfer of the three kinds of state, application-level, ORB/POA-level and infrastructure-level state, that are associated with each replicated object. Eternal's mechanisms for recovery include the synchronization of the the state retrieval and the state assignment messages, as well as the logging and replay of messages and checkpoints.

Abstract Multicast group communication is needed for fault‐tolerant distributed systems and, in particular, for the new Fault Tolerant CORBA standard, to maintain strong replica consistency. However, different multicast group communication protocols are appropriate for different environments, which makes it difficult to define a single standard multicast protocol. In this paper, we present a multicast group communication engine and bridge for CORBA that allows multiple group communication protocols to be used concurrently. We also present the Fault Tolerant Multicast Protocol, a group communication protocol that allows different fault tolerance systems to interoperate. The group communication engine and bridge places Lamport timestamps on messages, and multicasts messages to groups, using one or more group communication protocols. The group communication protocols reliably deliver the timestamped messages in timestamp order to the group communication engine, which integrates these streams of messages into a single stream for delivery in timestamp order. The Fault Tolerant Multicast Protocol operates over IP Multicast, and consists of the Reliable Multicast Protocol which provides reliable source‐ordered message delivery, the Reliable Ordered Multicast Protocol which provides reliable totally ordered message delivery, and the Host Group Membership Protocol which provides host group membership services. Copyright © 2001 John Wiley & Sons, Ltd.

The Eternal system supports networked enterprise applications that must operate continuously 24 hours per day, 7 days per week. Based on the CORBA standard, Eternal provides object replication not only for fault tolerance but also for live software upgrades, as well as resource management facilities. Through the use of interceptors, Eternal renders the object replication transparent to the application, as well as to the ORB and to the operating system. Thus, Eternal works with commercial off-the-shelf CORBA ORBs and standard unmodified operating systems. Eternal handles the difficult issues of object replication, fault tolerance, live upgrades and resource management, thereby allowing the application programmers to focus on the applications.

The Immune system aims to provide survivability to CORBA applications, enabling them to continue to operate despite malicious attacks, accidents or faults. Every object within the CORBA application is actively replicated by the Immune system, with majority voting applied on incoming invocations and responses to each replica of the object. Secure multicast protocols are employed to enable the majority voting to be effective, even when processors within the network and objects within the application become corrupted.

In CORBA-based applications that depend on object replication for fault tolerance, inconsistencies in the states of the replicas of an object can arise when concurrent threads within those replicas perform updates in different orders. By imposing a single logical thread of control on every replicated multithreaded CORBA client or server object, and by providing deterministic scheduling of threads and operations a cross the replicas of each object, the Eternal system achieves consistent object replication. The Eternal system does this transparently, with no modification to the application, the ORB, or the concurrency model employed by the ORB.