In this article, we propose Gatling, a framework that automatically finds performance attacks caused by insider attackers in large-scale message-passing distributed systems. In performance attacks, malicious nodes deviate from the protocol when sending or creating messages, with the goal of degrading system performance. We identify a representative set of basic malicious message delivery and lying actions and design a greedy search algorithm that finds effective attacks consisting of a subset of these actions. Although lying malicious actions are protocol dependent, requiring the format and meaning of messages, Gatling captures them without needing to modify the target system by using a type-aware compiler. We have implemented and used Gatling on nine systems, a virtual coordinate system, a distributed hash table lookup service and application, two multicast systems and one file sharing application, and three secure systems designed specifically to tolerate insiders, two based on virtual coordinates and one using Outlier Detection, one invariant derived from physical laws, and the last one a Byzantine resilient replication system. We found a total of 48 attacks, with the time needed to find each attack ranging from a few minutes to a few hours.

Code diversification has been proposed as a technique to mitigate code reuse attacks, which have recently become the predominant way for attackers to exploit memory corruption vulnerabilities. As code reuse attacks require detailed knowledge of where code is in memory, diversification techniques attempt to mitigate these attacks by randomizing what instructions are executed and where code is located in memory. As an attacker cannot read the diversified code, it is assumed he cannot reliably exploit the code.

Security and performance are critical goals for distributed systems. The increased design complexity, incomplete expertise of developers, and limited functionality of existing testing tools often result in bugs and vulnerabilities that prevent implementations from achieving their design goals in practice. Many of these bugs, vulnerabilities, and misconfigurations manifest after the code has already been deployed making the debugging process difficult and costly. In this paper, we present Turret, a platform for automatically finding performance attacks in unmodified implementations of distributed systems. Turret does not require the user to provide any information about vulnerabilities and runs the implementation in the same operating system setup as the deployment, with an emulated network. Turret uses a new attack finding algorithm and several optimizations that allow it to find attacks in a matter of minutes. We ran Turret on 5 different distributed system implementations specifically designed to tolerate insider attacks, and found 30 performance attacks, 24 of which were not previously reported to the best of our knowledge.

Virtual coordinate systems (VCSs) provide accurate estimations of latency between arbitrary hosts on a network, while conducting a small amount of actual measurements and relying on node cooperation. While these systems have good accuracy under benign settings, they suffer a severe decrease of their effectiveness when under attack by compromised nodes acting as insider attackers. Previous defenses mitigate such attacks by using machine learning techniques to differentiate good behavior (learned over time) from bad behavior. However, these defense schemes have been shown to be vulnerable to advanced attacks that make the schemes learn malicious behavior as good behavior. We present Newton, a decentralized VCS that is robust to a wide class of insider attacks. Newton uses an abstraction of a real-life physical system, similar to that of Vivaldi, but in addition uses safety invariants derived from Newton's laws of motion. As a result, Newton does not need to learn good behavior and can tolerate a significantly higher percentage of malicious nodes. We show through simulations and real-world experiments on the PlanetLab testbed that Newton is able to mitigate all known attacks against VCSs while providing better accuracy than Vivaldi, even in benign settings. Finally, we show how to design a VCS that better matches a real physical system, thus allowing for more intuitive and tighter system parameters that are even more difficult to exploit by attackers.

Virtual coordinate systems (VCS) provide accurate estimations of latency between arbitrary hosts on a network, while conducting a small amount of actual measurements and relying on node cooperation. While these systems have good accuracy under benign settings, they suffer a severe decrease of their effectiveness when under attack by compromised nodes acting as insider attackers. Previous defenses mitigate such attacks by using machine learning techniques to differentiate good behavior (learned over time) from bad behavior. However, these defense schemes have been shown to be vulnerable to advanced attacks that make the schemes learn malicious behavior as good behavior. We present Newton, a decentralized VCS that is robust to a wide class of insider attacks. Newton uses an abstraction of a real-life physical system, similar to that of Vivaldi, but in addition uses safety invariants derived from Newton's laws of motion. As a result, Newton does not need to learn good behavior and can tolerate a significantly higher percentage of malicious nodes. We show through simulations and real-world experiments on the Planet Lab test bed that Newton is able to mitigate all known attacks against VCS while providing better accuracy than Vivaldi, even in benign settings.

We conduct a detailed simulation study to examine how localizing P2P traffic within network boundaries impacts the profitability of an ISP. A distinguishing aspect of our work is the focus on Internet-wide implications, i.e., how adoption of localization within an ISP affects both itself and other ISPs. Our simulations are based on detailed models that estimate inter-autonomous-system (AS) P2P traffic and inter-AS routing, localization models that predict the extent to which P2P traffic is reduced, and pricing models that predict the impact of changes in traffic on the profit of an ISP. We evaluate our models by using a large-scale crawl of BitTorrent containing over 138 million users sharing 2.75 million files. Our results show that the benefits of localization must not be taken for granted. Some of our key findings include: 1) residential ISPs can actually lose money when localization is employed, and some of them will not see increased profitability until other ISPs employ localization; 2) the reduction in costs due to localization will be limited for small ISPs and tends to grow only logarithmically with client population; and 3) some ISPs can better increase profitability through alternate strategies to localization by taking advantage of the business relationships they have with other ISPs.

Virtual coordinate systems provide an accurate and efficient service that allows hosts on the Internet to determine latency to arbitrary hosts based on information provided by a subset of participating nodes. Unfortunately, the accuracy of the service can be severely impacted by compromised nodes providing misleading information. We define and use a game theory framework in order to identify the best attack and defense strategies assuming that the attacker is aware of the defense mechanisms. Our approach leverages concepts derived from the Nash equilibrium to model more powerful adversaries. We consider attacks that target the latency estimation (inflation, deflation, oscillation) and defense mechanisms that combine outlier detection with control theory to deter adaptive adversaries. We apply the game theory framework to demonstrate the impact and efficiency of these attack and defense strategies using a well-known virtual coordinate system and real-life Internet data sets.

We compare two representative streaming systems using mesh-based and multiple tree-based overlay routing through deployments on the PlanetLab wide-area experimentation platform. To the best of our knowledge, this is the first study to compare streaming overlay architectures in real Internet settings, considering not only intuitive aspects such as scalability and performance under churn, but also less studied factors such as bandwidth and latency heterogeneity of overlay participants. Overall, our study indicates that mesh-based systems are superior for nodes with high bandwidth capabilities and low round trip times, while multi-tree based systems currently cope better with stringent real time deadlines under heterogeneous conditions.