This paper presents an empirical investigation of security vulnerabilities in four categories of transportation IoT devices: Electric Vehicle (EV) Chargers, EVs, non-EVs, and Other Traffic Devices. The results are based on data extracted from the Common Vulnerabilities and Exposures (CVEs) reported in the National Vulnerability Database (NVD). We analyzed 159 CVEs and explored the CWE (Common Weakness Enumeration) and CVSS (Common Vulnerability Scoring System) information associated with them. Our results showed that the security vulnerabilities were distributed unevenly across the 23 vulnerability classes, with the top six most common classes accounting for 74% to 89% of all vulnerabilities in each category of IoT devices. EV Chargers had the highest mean and median CVSS severity score, followed by Other Traffic Devices and EVs. Non-EVs had the lowest CVSS, which were statistically significantly different than the other categories of transportation IoT devices. The paper also presents the lessons learned and the practical implications of our empirical findings.

Reusable software libraries, frameworks, and components, such as those provided by open source ecosystems and third-party suppliers, accelerate digital innovation. However, recent years have shown almost exponential growth in attackers leveraging these software artifacts to launch software supply chain attacks. Past well-known software supply chain attacks include the SolarWinds, log4j, and xz utils incidents. Supply chain attacks are considered to have three major attack vectors: through vulnerabilities and malware accidentally or intentionally injected into open source and third-party dependencies/components/containers ; by infiltrating the build infrastructure during the build and deployment processes; and through targeted techniques aimed at the humans involved in software development, such as through social engineering. Plummeting trust in the software supply chain could decelerate digital innovation if the software industry reduces its use of open source and third-party artifacts to reduce risks. This article contains perspectives and knowledge obtained from intentional outreach with practitioners to understand their practical challenges and from extensive research efforts. We then provide an overview of current research efforts to secure the software supply chain. Finally, we propose a future research agenda to close software supply chain attack vectors and support the software industry.

Existing research has primarily delved into the realm of computer science outreach aimed at K-12 students, with a focus on both informal and non-formal approaches. However, a noticeable research gap exists when it comes to cybersecurity outreach tailored specifically for underserved secondary school students. This article addresses this void by presenting an iterative pilot of a cybersecurity curriculum. This innovative curriculum integrates a one-week summer camp and a series of 1.5-hour workshops designed to provide students with a comprehensive understanding of cybersecurity.

OBJECTIVE: Firearm injury research necessitates using data from often-exploited vulnerable populations of Black and Brown Americans. In order to reduce bias against protected attributes, this study provides a theoretical framework for establishing trust and transparency in the use of AI with the general population. METHODS: We propose a Model Facts template that is easily extendable and decomposes accuracy and demographics into standardized and minimally complex values. This framework allows general users to assess the validity and biases of a model without diving into technical model documentation. EXAMPLES: We apply the Model Facts template on 2 previously published models, a violence risk identification model and a suicide risk prediction model. We demonstrate the ease of accessing the appropriate information when the data are structured appropriately. DISCUSSION: The Model Facts template is limited in its current form to human based data and biases. Like nutrition facts, it will require educational programs for users to grasp its full utility. Human computer interaction experiments should be conducted to ensure model information is communicated accurately and in a manner that improves user decisions. CONCLUSION: The Model Facts label is the first framework dedicated to establishing trust with end users and general population consumers. Implementation of Model Facts into firearm injury research will provide public health practitioners and those impacted by firearm injury greater faith in the tools the research provides.

Vulnerabilities with publically known exploits typically form 2-7% of all vulnerabilities reported for a given software version. With a smaller number of known exploited vulnerabilities compared with the total number of vulnerabilities, it is more difficult to model and predict when a vulnerability with a known exploit will be reported. In this paper, we introduce an approach for predicting the discovery pattern of publically known exploited vulnerabilities using all publically known vulnerabilities reported for a given software. Eight commonly used vulnerability discovery models (VDMs) and one neural network model (NNM) were utilized to evaluate the prediction capability of our approach. We compared their predictions results with the scenario when only exploited vulnerabilities were used for prediction. Our results show that, in terms of prediction accuracy, out of eight software we analyzed, our approach led to more accurate results in seven cases. Only in one case, the accuracy of our approach was worse by 1.6%.

In this paper, we correlate human characteristics with cyber security behavior intentions. While previous papers have identified correlations between certain human traits and specific cyber security behavior intentions, we present a comprehensive study that examines how risk-taking preferences, decision-making styles, demographics, and personality traits influence the security behavior intentions of device securement, password generation, proactive awareness, and updating. To validate and expand the work of Egelman and Peer, we conducted a survey of 369 students, faculty, and staff at a large public university and found that individual differences accounted for 5%–23% of the variance in cyber security behavior intentions. Characteristics such as financial risk-taking, rational decision-making, extraversion, and gender were found to be significant unique predictors of good security behaviors. Our study revealed both validations and contradictions of related work in addition to finding previously unreported correlations. We motivate the importance of studies such as ours by demonstrating how the influence of individual differences on security behavior intentions can be environment-specific. Thus, some security decisions should also depend on the environment.

We present AVAMAT: AntiVirus and Malware Analysis Tool - a tool for analysing the malware detection capabilities of AntiVirus (AV) products running on different operating system (OS) platforms. Even though similar tools are available, such as VirusTotal and MetaDefender, they have several limitations, which motivated the creation of our own tool. With AVAMAT we are able to analyse not only whether an AV detects a malware, but also at what stage of inspection does it detect it and on what OS. AVAMAT enables experimental campaigns to answer various research questions, ranging from the detection capabilities of AVs on OSs, to optimal ways in which AVs could be combined to improve malware detection capabilities.

There are different ways for malware to spread from device to device. Some methods depend on the presence of a vulnerability that can be exploited along with some action taken by a user of the device. Malware propagating through email are one such example. While existing research has explored potential factors and models for simulating this form of propagation, it remains for these potential factors and models to be empirically tested and supported using field collected incident data. We review a common model for simulating the spread of email malware and use simulations to illustrate the potential impacts of connection topologies and different distributions of associated user actions. We use simulations to examine the potential impact of two types of commonly available interventions-patching vulnerable devices and blocking the transmission of infected messages in combination with different connection topologies and different distributions of user actions. Finally, we explore the use of Approximate Bayesian Computation (ABC) as a method to compare simulation results to empirical data to assess different model features, and to infer corresponding model parameter values from field collected email malware incident data.

There are currently no generally accepted metrics for information security issues. One reason is the lack of validation using empirical data. In this practical experience report, we investigate whether metrics obtained from security devices used to monitor network traffic can be employed as indicators of security incidents. If so, security experts can use this information to better define priorities on security inspection and also to develop new rules for incident prevention. The metrics we investigate are derived from intrusion detection and prevention system (IDPS) alert events. We performed an empirical case study using IDPS data provided by a large organization of about 40,000 computers. The results indicate that characteristics of alerts can be used to depict trends in some security issues and consequently serve as indicators of security performance.

AntiVirus (AV) products use multiple components to detect malware. A component which is found in virtually all AVs is the signature-based detection engine: this component assigns a particular signature label to a malware that the AV detects. In previous analysis [1–3], we observed cases of regressions in several different AVs: i.e. cases where on a particular date a given AV detects a given malware but on a later date the same AV fails to detect the same malware. We studied this aspect further by analyzing the only externally observable behaviors from these AVs, namely whether AV engines detect a malware and what labels they assign to the detected malware. In this paper we present the results of the analysis about the relationship between the changing of the labels with which AV vendors recognize malware and the AV regressions.

Computer and network security incidents have financial and other consequences to organizations, such as direct business losses from theft of proprietary information or from just reputational damage. There are also costs for restoring operations and protecting against threats. Being able to quantify the impact of different factors within an organization may provide additional context for prevention and remediation efforts. This paper examines a large set of security incident data along with some population characteristic data from an organization's network. We discuss the rationale for examining the different population characteristics and their potential influence on computer security incidents. We then create logistic regression models using the population characteristics to forecast which machines in the population may be involved in a computer security incident. We evaluate the models using the forecasts as a set of unequal probability weights combined with repeated sampling. We also explore different time windows used for the inclusion of data during model creation.

This paper describes an empirical study to assess whether computer focused crimes are impacted by system configurations. The study relies on data collected during 30 days on a farm of target computers of various configurations (disk space, memory size, and bandwidth). In addition, some target computers included a warning sign. Following a brute force attack on SSH, attackers randomly access one of these computer configurations and are allowed to use it for 30 days. We monitor network traffic and attackers' keystrokes to analyze the attack. This paper focuses specifically on the crime, i.e., the use of the computer to launch an attack towards an external target. We define various computer focused crime characteristics (i.e., whether the attack was destructive or not, whether the target was an opportunity or a choice, whether the attack was coordinated or not) and analyze whether the committed crime is significantly impacted by the system configuration.

Many office devices have a history of being networked (such as printers) and others without the same past are increasingly becoming networked (such as photocopiers). The modern networked versions of previously non-networked devices have much in common with traditional networked servers in terms of features and functions. While an organization may have policies and procedures for securing traditional network servers, securing networked office devices providing similar services can easily be overlooked. In this paper we present an evaluation of privacy and security risks found when examining over 1,800 networked office devices connected to a large university network. We use the STRIDE threat model to categorize threats and vulnerabilities and then we group the devices according to assessed risk from the perspective of the university. We found that while steps had been taken to secure some devices, many were using default or unsecured configurations.

This paper describes an empirical research study to characterize attackers and attacks against targets of opportunity. A honey net infrastructure was built and deployed over 167 days that leveraged three different honey pot configurations and a SSH-based authentication proxy to attract and follow attackers over several weeks. A total of 211 attack sessions were recorded and evidence was collected at each stage of the attack sequence: from discovery to intrusion and exploitation of rogue software. This study makes two important contributions: 1) we introduce a new approach to measure attacker skills, and 2) we leverage keystroke profile analysis to differentiate attackers beyond their IP address of origin.

This practical experience report presents the results of an experiment aimed at understanding the sequence of malicious actions following a remote compromise. The type of rogue software installed during attacks was used to classify and understand sequences of malicious actions. For this experiment, we used four Linux target computers running SSH with simple passwords. During the eight-month data collection period, we recorded a total of 1,171 attack sessions. In these sessions, attackers typed a total of 20,335 commands that we categorized into 24 specific actions. These actions were analyzed based on the type of rogue software installed by attackers.

Monitoring filesystem data is a common method used to detect intrusions. Once a computer is compromised, an attacker may alter files, add new files or delete existing files. The changes that attackers make may target any part of the filesystem, including metadata along with files (e.g., permissions, ownerships and inodes). The accuracy of detecting an intrusion depends on the data audited: if an intrusion does not manifest in the data, the intrusion will not be detected. Moreover, not all files, which contain filesystem activity, are suitable to detect intrusions, as some may fail to provide useful information. In this paper, we describe an empirical study that focused on filesystem attack activity after a SSH compromise. Three types of attacker action are considered: reconnaissance, password modification, and malware download. For each type of action, we evaluated the files to audit using metrics derived from the field of information theory and estimated with the empirical SSH compromise data.

Organizations face increasing challenges in addressing and preventing computer and network security incidents. There are financial consequences from security incidents. These include lost time and resources used during recovery, possible theft of personal and/or proprietary information, and reputational damage that may negatively impact stock prices or reduce consumer confidence in a company. Being able to understand and predict trends in computer and network security incidents can aid an organization with resource allocation for prevention of such incidents, as well as evaluation of mitigation strategies. We look at using time series models with a large set of security incident data. We examine appropriateness of the data for modeling and consider needed transformations. Parameter search and model selection criteria are discussed. Then, forecasts from time series models are compared to forecasts from Non-Homogeneous Poisson Process (NHPP) software reliability growth (SRG) models.

Abstract There are many group communication systems (GCSs) that provide consistent group membership and reliable, ordered multicast properties in the presence of crash faults. However, relatively few GCS implementations are able to provide these properties in the presence of malicious faults resulting from intrusions. We describe the systematic transformation of a crash‐tolerant GCS, namely C‐Ensemble, into an intrusion‐tolerant GCS, the ITUA GCS. To perform the transformation, we devised intrusion‐tolerant versions of key group communication protocols. We then inserted implementations of the protocols into C‐Ensemble and made significant changes to the rest of the C‐Ensemble protocol stack to make the stack intrusion tolerant. We quantify the cost of providing intrusion‐tolerant group communication in two ways. First, we quantify the implementation effort by presenting a detailed analysis of the amount of change required to the original C‐Ensemble system. In doing so, we provide insight into the choice of building an intrusion‐tolerant GCS from scratch versus building one by leveraging a crash‐tolerant implementation. Second, we quantify the run‐time performance cost of tolerating intrusions by presenting results from an experimental evaluation of the main intrusion‐tolerant microprotocols. The results are analyzed to identify the parts that contribute the most overhead while providing intrusion tolerance during both normal operation and recovery from intrusions. Copyright © 2007 John Wiley & Sons, Ltd.

This practical experience report presents the results of an experiment aimed at building a profile of attacker behavior following a remote compromise. For this experiment, we utilized four Linux honeypot computers running SSH with easily guessable passwords. During the course of our research, we also determined the most commonly attempted usernames and passwords, the average number of attempted logins per day, and the ratio of failed to successful attempts. To build a profile of attacker behavior, we looked for specific actions taken by the attacker and the order in which they occurred. These actions were: checking the configuration, changing the password, downloading a file, installing/running rogue code, and changing the system configuration.

Computer and network security incidents have increasing financial consequences as demand for network accessibility and connectivity to resources continues to rise. These security incidents can lead to direct financial losses either through data theft of personal and/or proprietary information as well as a reputational damage which may negatively impact stock prices or consumer confidence in a company. This paper examines a large set of security incident data using tools from the software reliability community. We look at applying Non-Homogenous Poisson Process (NHPP) models as a method for describing the reliability growth process. We examine the full set of incidents as well as subsets of the data based on incident types. We look at using the Laplace test to guide selection of the appropriate models. Then, based on the trend results, we apply various NHPP models (i.e., Goel-Okumutu, S-Shaped, Duane, and K-Stage Curve) to illustrate the relevance of using these models to fit the incident data and to predict future incidents.

This paper empirically compares malicious traffic originating inside an organization (i.e., internal traffic) with malicious traffic originating outside an organization (i.e., external traffic). Two honeypot target computers were deployed to collect malicious traffic data over a period of fifteen weeks. In the first study we showed that there was a weak correlation between internal and external traffic based on the number of malicious connections. Since the type of malicious activity is linked to the port that was targeted, we focused on the most frequently targeted ports. We observed that internal malicious traffic often contained different malicious content compared to that of external traffic. In the third study, we discovered that the volume of malicious traffic was linked to the day of the week. We showed that internal and external malicious activities differ: where the external malicious activity is quite stable over the week, the internal traffic varied as a function of the users' activity profile.

To support decision-making for security-risk mitigation and the appropriate selection of security countermeasures, we propose a system dynamics model of the security aspects of an enterprise system. We developed such an executable model, incorporating the concept of archetypes. We present here one archetype for computer security, namely symptomatic fixes (or shifting the burden). Using simulation, we show one instance of how this archetype can be used for recognizing and diagnosing typical situations, as well as for fixing problems. The global effects of changes and behavioral trends are examined, and other instances of symptomatic fixes in security are described as well

This paper analyzes malicious activity collected from a test-bed, consisting of two target computers dedicated solely to the purpose of being attacked, over a 109 day time period. We separated port scans, ICMP scans, and vulnerability scans from the malicious activity. In the remaining attack data, over 78% (i.e., 3,677 attacks) targeted port 445, which was then statistically analyzed. The goal was to find the characteristics that most efficiently separate the attacks. First, we separated the attacks by analyzing their messages. Then we separated the attacks by clustering characteristics using the K-Means algorithm. The comparison between the analysis of the messages and the outcome of the K-Means algorithm showed that 1) the mean of the distributions of packets, bytes and message lengths over time are poor characteristics to separate attacks and 2) the number of bytes, the mean of the distribution of bytes and message lengths as a function of the number packets are the best characteristics for separating attacks

Empirical evaluation of dependability is a complement to modeling and analytical methods. Although empirical evaluation applies to real systems and is more realistic, more accurate, and provides a higher level of confidence, it is not extensively used, for multiple reasons: it is time and effort consuming; the results have limited portability; there are no benchmarks to define what measures to be collected, how to measure the data, how to report it, what are the models that indicate the ranges of "good" and "bad" values, and how to compare the values for different systems. Due to these challenges, very few results and empirical data are available for dependability evaluation, comparison, and benchmarking. Acknowledging these challenges, but also the need for empirical dependability evaluation, this workshop will bring together researchers and practitioners for sharing their solutions and results, as well as for documenting their needs, problems, and ideas with respect to this topic. The workshop will include presentations and a hands-on session where participants will work on finding solutions to pre-selected questions.

This practical experience report presents the results of an investigation into the threat of attacks associated with the chat medium IRC. A combination of simulated users (i.e., bots), some configured with scripts that simulated conversations, and regular users were used. The average number of attacks per day a user on IRC can expect, the effect of channel activity, gender based on the name, and network type on the number of attacks were determined. The social structure of IRC channels and the types of users that use it were analyzed. The results indicate that attacks through IRC channels come from human users selecting targets rather than automated scripts targeting every user in a channel

Abstract Applications that are part of a mission‐critical information system need to maintain a usable level of key services through ongoing cyber‐attacks. In addition to the well‐publicized denial of service (DoS) attacks, these networked and distributed applications are increasingly threatened by sophisticated attacks that attempt to corrupt system components and violate service integrity. While various approaches have been explored to deal with DoS attacks, corruption‐inducing attacks remain largely unaddressed. We have developed a collection of mechanisms based on redundancy, Byzantine fault tolerance, and adaptive middleware that help distributed, object‐based applications tolerate corruption‐inducing attacks. In this paper, we present the ITUA architecture, which integrates these mechanisms in a framework for auto‐adaptive intrusion‐tolerant systems, and we describe our experience in using the technology to defend a critical application that is part of a larger avionics system as an example. We also motivate the adaptive responses that are key to intrusion tolerance, and explain the use of the ITUA architecture to support them in an architectural framework. Copyright © 2006 John Wiley & Sons, Ltd.

This paper describes an experimental approach to determine the correlation between port scans and attacks. Discussions in the security community often state that port scans should be considered as precursors to an attack. However, very few studies have been conducted to quantify the validity of this hypothesis. In this paper, attack data were collected using a test-bed dedicated to monitoring attackers. The data collected consist of port scans, ICMP scans, vulnerability scans, successful attacks and management traffic. Two experiments were performed to validate the hypothesis of linking port scans and vulnerability scans to the number of packets observed per connection. Customized scripts were then developed to filter the collected data and group them on the basis of scans and attacks between a source and destination IP address pair. The correlation of the filtered data groups was assessed. The analyzed data consists of forty-eight days of data collection for two target computers on a heavily utilized subnet.

Evaluation of computing system security requires knowledge of the vulnerabilities present in the system and of potential attacks against the system. Vulnerabilities can be classified based on their location as application vulnerabilities, network vulnerabilities, or host vulnerabilities. This paper describes Ferret-Windows, a new software tool for checking host vulnerabilities on the Windows platforms. This tool helps system administrators by quickly finding vulnerabilities that are present on a host. It is designed and implemented in a modular way: a plug-in module is used for each vulnerability checked, and each possible output format is specified by a plug-in module. Moreover, several vulnerability fixing plug-in modules exist to help users remove specific vulnerabilities. As a result, Ferret-Windows is extensible, and can easily be kept up-to-date through the addition of checks for new vulnerabilities as they are identified. Finally, Ferret-Windows is a freely available open-source software.

Evaluation of computing system security requires knowledge of the vulnerabilities present in the system and of potential attacks against the system. Vulnerabilities can be classified based on their location as application vulnerabilities, network vulnerabilities, or host vulnerabilities. We describe Ferret, a new software tool for checking host vulnerabilities. Ferret helps system administrators by quickly finding vulnerabilities that are present on a host. It is designed and implemented in a modular way: a different plug-in module is used for each vulnerability checked, and each possible output format is specified by a plug-in module. As a result, Ferret is extensible, and can easily be kept up-to-date through addition of checks for new vulnerabilities as they are discovered; the modular approach also makes it easy to provide specific configurations of Ferret tailored to specific operating systems or use environments. Ferret is a freely available open-source software implemented in Perl.

Validation of the dependability of distributed systems via fault injection is gaining importance because distributed systems are being increasingly used in environments with high dependability requirements. The fact that distributed systems can fail in subtle ways that depend on the state of multiple parts of the system suggests that a global-state-based fault injection mechanism should be used to validate them. However, global-state-based fault injection is challenging since it is very difficult in practice to maintain the global state of a distributed system at runtime with minimal intrusion into the system execution. We present Loki, a global-state-based fault injector, which has been designed with the goals of low intrusion, high precision, and high flexibility. Loki achieves these goals by utilizing the ideas of partial view of global state, optimistic synchronization, and offline analysis. In Loki, faults are injected based on a partial, view of the global state of the system, and a post-runtime analysis is performed to place events and injections into a single global timeline and to discard experiments with incorrect fault injections. Finally, the experiments with correct fault injections are used to estimate user-specified performance and dependability measures. A flexible measure language has been designed that facilitates the specification of a wide range of measures.

As computer systems become more complex and more widely distributed, it is becoming increasingly difficult to remove all vulnerabilities that can potentially be exploited by intruders. Intrusion tolerance is an emerging approach that aims to enable systems to continue functioning in spite of successful intrusions. Before intrusion tolerance is accepted as an approach to security, there must be quantitative techniques to measure its efficacy. However, there have been very few attempts at quantitative validation of intrusion-tolerant systems or, for that matter, of security in general. In this paper, we show that probabilistic validation through stochastic modeling is an attractive mechanism for evaluating intrusion tolerance. We demonstrate our approach by using stochastic activity networks to quantitatively validate an intrusion-tolerant replication management system. We characterize the intrusion tolerance provided by the system through several measures defined on the model, and study variations in these measures in response to changes in system parameters to evaluate the relative merits of various design choices.

Experimental evaluation is an important way to assess distributed systems, and fault injection is the dominant technique in this area for the evaluation of a system's dependability. For distributed systems, network failure is an important fault model. Physical network failures often have far-reaching effects, giving rise to multiple correlated failures as seen by higher-level protocols. This paper presents an experimental evaluation, using the Loki fault injector, which provides insight into the impact that correlated network partitions have on the Coda distributed file system. In this evaluation, Loki created a network partition between two Coda file servers, during which updates were made at each server to the same replicated data volume. Upon repair of the partition, a client requested directory resolution to converge the diverging replicas. At various stages of the resolution, Loki invoked a second correlated network partition, thus allowing us to evaluate its impact on the system's correctness, performance, and availability.

Building dependable distributed systems from commercial off-the-shelf components is of growing practical importance. For both cost and production reasons, there is interest in approaches and architectures that facilitate building such systems. The AQuA architecture is one such approach; its goal is to provide adaptive fault tolerance to CORBA applications by replicating objects. The AQuA architecture allows application programmers to request desired levels of dependability during applications' runtimes. It provides fault tolerance mechanisms to ensure that a CORBA client can always obtain reliable services, even if the CORBA server object that provides the desired services suffers from crash failures and value faults. AQuA includes a replicated dependability manager that provides dependability management by configuring the system in response to applications' requests and changes in system resources due to faults. It uses Maestro/Ensemble to provide group communication services. It contains a gateway to intercept standard CORBA IIOP messages to allow any standard CORBA application to use AQuA. It provides different types of replication schemes to forward messages reliably to the remote replicated objects. All of the replication schemes ensure strong, data consistency among replicas. This paper describes the AQuA architecture and presents, in detail, the active replication pass-first scheme. In addition, the interface to the dependability manager and the design of the dependability manager replication are also described. Finally, we describe performance measurements that were conducted for the active replication pass-first scheme, and we present results from our study of fault detection, recovery, and blocking times.

A dependable middleware should be able to adaptively share the distributed resources it manages in order to meet diverse application requirements, even when the quality of service (QoS) is degraded due to uncertain variations in load and unanticipated failures. We have addressed this issue in the context of a dependable middleware that adaptively manages replicated servers to deliver a timely and consistent response to time-sensitive client applications. These applications have specific temporal and consistency requirements, and can tolerate a certain degree of relaxed consistency in exchange for better response time. We propose a flexible QoS model that allows clients to specify their timeliness and consistency constraints. We also propose an adaptive framework that dynamically selects replicas to service a client's request based on the prediction made by probabilistic models. These models use the feedback from online performance monitoring of the replicas to provide probabilistic guarantees for meeting a client's QoS specification. The experimental results we have obtained demonstrate the role of feedback and the efficacy of simple analytical models for adaptively sharing the available replicas among the users under different workload scenarios.

One well-known challenge in using replication to service multiple clients concurrently is that of delivering a timely and consistent response to the clients. In this paper, we address this problem in the context of client applications that have specific temporal and consistency requirements. These applications can tolerate a certain degree of relaxed consistency, in exchange for better response time. We propose a flexible QoS model that allows these clients to specify their temporal and consistency constraints. In order to select replicas to serve these clients, we need to control of the inconsistency of the replicas, so that we have a large enough pool of replicas with the appropriate state to meet a client's timeliness, consistency, and dependability requirements. We describe an adaptive framework that uses lazy update propagation to control the replica inconsistency and employs a probabilistic approach to select replicas dynamically to service a client, based on its QoS specification. The probabilistic approach predicts the ability of a replica to meet a client's QoS specification by using the performance history collected by monitoring the replicas at runtime. We conclude with experimental results based on our implementation.

Group communication systems that provide consistent group membership and reliable, ordered multicast properties in the presence of faults resulting from malicious intrusions have not been analyzed extensively to quantify the cost of tolerating these intrusions. This paper attempts to quantify this cost by presenting results from an experimental evaluation of three new intrusion-tolerant microprotocols that have been added to an existing crash-fault-tolerant group communication system. The results are analyzed to identify the parts that contribute the most overhead during provision of intrusion tolerance at the group communication system level.

We describe a group membership protocol that is part of an intrusion-tolerant group communication system, and present an effort to use formal tools to model and validate our protocol. We describe in detail the most difficult part of the validation exercise, which was the determination of the right level of abstraction of the protocol for formally specifying the protocol. The validation exercise not only formally showed that the protocol satisfies its correctness claims, but also provided information that will help us make the protocol more efficient without violating correctness.

Building large-scale distributed object-oriented systems that provide multidimensional quality of service (QoS) in terms of fault tolerance, scalability, and performance is challenging. In order to meet this challenge, we need an architecture that can ensure that applications' requirements can be met while providing reusable technologies and software solutions. This paper describes techniques, based on the AQuA architecture, that enhance the applications' dependability and scalability by introducing two types of group members and a novel passive replication scheme. In addition, we describe how to make the management structure itself dependable by using the passive replication scheme. Finally, we provide performance measurements for the passive replication scheme.

Server replication is commonly used to improve the fault tolerance and response time of distributed services. An important problem when executing time-critical applications in a replicated environment is that of preventing timing failures by dynamically selecting the replicas that can satisfy a client's timing requirement, even when the quality of service is degraded due to replica failures and excess load on the server. We describe the approach we have used to solve this problem in AQuA, a CORBA-based middleware that transparently replicates objects across a local area network. The approach we use estimates a replica's response time distribution based on performance measurements regularly broadcast by the replica. An online model uses these measurements to predict the probability with which a replica can prevent a timing failure for a client. A selection algorithm then uses this prediction to choose a subset of replicas that can together meet the client's timing constraints with at least the probability requested by the client. We conclude with experimental results based on our implementation.

The AQuA architecture provides adaptive fault tolerance to CORBA applications by replicating objects and providing a high-level method that an application can use to specify its desired level of dependability. This paper presents the algorithms that AQUA uses, when an application's dependability requirements can change at runtime, to tolerate both value faults in applications and crash failures simultaneously. In particular, we provide an active replication communication scheme that maintains data consistency among replicas, detects crash failures, collates the messages generated by replicated objects, and delivers the result of each vote. We also present an adaptive majority voting algorithm that enables the correct ongoing vote while both the number of replicas and the majority size dynamically change. Together, these two algorithms form the basis of the mechanism for tolerating and recovering from value faults and crash failures in AQuA.

Distributed applications can fail in subtle ways that depend on the state of multiple parts of a system. This complicates the validation of such systems via fault injection, since it suggests that faults should be injected based on the global state of the system. In Loki, fault injection is performed based on a partial view of the global state of a distributed system, i.e. faults injected in one node of the system can depend on the state of other nodes. Once faults are injected, a post-runtime analysis, using off-line clock synchronization, is used to place events and injections on a single global timeline and to determine whether the intended faults were properly injected. Finally, experiments containing successful fault injections are used to estimate the specified measures. In addition to briefly reviewing the concepts behind Loki and its organization, we detail Loki's user interface. In particular, we describe the graphical user interfaces for specifying state machines and faults, for executing a campaign and for verifying whether the faults were properly injected.

Validation of distributed systems using fault injection is difficult because of their inherent complexity, lack of a global clock, and lack of an easily accessible notion of a global state. To address these challenges, the Loki fault injector injects faults based on a partial view of the global state of a distributed system, and performs a post-runtime analysis using an off-line clock synchronization algorithm to determine whether the faults were properly injected. In this paper, we first describe an enhanced runtime architecture for the Loki fault injector and then present a new method for obtaining measures in Loki. The enhanced runtime allows dynamic entry and exit of nodes in the system. It also offers more efficient multicast of notification messages and more efficient communication between state machines on the same host, and is more scalable than the previous runtime. We then detail a new and flexible method for obtaining a wide range of performance and dependability measures in Loki.

This paper describes the basis for and preliminary implementation of a new fault injector, called Loki, developed specifically for distributed systems. Loki addresses issues related to injecting correlated faults in distributed systems. In Loki, fault injection is performed based on a partial view of the global state of an application. In particular, facilities are provided to pass user-specified state information between nodes to provide a partial view of the global state in order to try to inject complex faults successfully. A post-runtime analysis, using an off-line clock synchronization and a bounding technique, is used to place events and injections on a single global time-line and determine whether the intended faults were properly injected. Finally, observations containing successful fault injections are used to estimate specified dependability measures. In addition to describing the details of our new approach, we present experimental results obtained from a preliminary implementation in order to illustrate Loki's ability to inject complex faults predictably.

This paper addresses the problem of estimating fault tolerance coverage through statistical processing of observations collected in fault-injection experiments. In an earlier paper, venous estimators based on simple sampling in the complete fault/activity input space and stratified sampling in a partitioned space were studied; frequentist confidence limits were derived based on a normal approximation. In this paper, the validity of this approximation is analyzed. The theory of confidence regions is introduced to estimate coverage without approximation when stratification is used. Three statistics are considered for defining confidence regions. It is shown that one-a vectorial statistic-is often more conservative than the other two. However, only the vectorial statistic is computationally tractable. We then consider Bayesian estimation methods for stratified sampling. Two methods are presented to obtain an approximation of the posterior distribution of the coverage by calculating its moments. The moments are then used to identify the type of the distribution in the Pearson distribution system, to estimate its parameters, and to obtain the coverage confidence limit. Three hypothetical example systems are used to compare the validity and the conservatism of the frequentist and Bayesian estimations.

Dependable distributed systems are difficult to build. This is particularly true if they have dependability requirements that change during the execution of an application, and are built with commercial off-the-shelf hardware. In that case, fault tolerance must be achieved using middleware software, and mechanisms must be provided to communicate the dependability requirements of a distributed application to the system and to adapt the system's configuration to try to achieve the desired dependability. The AQuA architecture allows distributed applications to request a desired level of availability using the Quality Objects (QuO) framework and includes a dependability manager that attempts to meet requested availability levels by configuring the system in response to outside requests and changes in system resources due to faults. The AQuA architecture uses the QuO runtime to process and invoke availability requests, the Proteus dependability manager to configure the system in response to faults and availability requests, and the Ensemble protocol stack to provide group communication services. Furthermore, a CORBA interface is provided to application objects using the AQuA gateway. The gateway provides a mechanism to translate between process-level communication, as supported by Ensemble, and IIOP messages, understood by Object Request Brokers. Both active and passive replication are supported, and the replication type to use is chosen based on the performance and dependability requirements of particular distributed applications.

Consensus protocols are used in a variety of reliable distributed systems, including both safety-critical and business-critical applications. The correctness of a consensus protocol is usually shown, by making assumptions about the environment in which it executes, and then proving properties about the protocol. But proofs about a protocol's behavior are only as good as the assumptions which were made to obtain them, and violation of these assumptions can lead to unpredicted and serious consequences. We present a new approach for the probabilistic verification of synchronous round based consensus protocols. In doing so, we make stochastic assumptions about the environment in which a protocol operates, and derive probabilities of proper and non proper behavior. We thus can account for the violation of assumptions made in traditional proof techniques. To obtain the desired probabilities, the approach enumerates possible states that can be reached during an execution of the protocol, and computes the probability of achieving the desired properties for a given fault and network environment. We illustrate the use of this approach via the evaluation of a simple consensus protocol operating under a realistic environment which includes performance, omission, and crash failures.