The University of Illinois at Urbana Champaign (Illinois), Pacific Northwest National Labs (PNNL), and the University of Southern California Information Sciences Institute (USC-ISI) consortium is working toward providing tools and expertise to enable collaborative research to improve security and resiliency of cyber physical systems. In this extended abstract we discuss the challenges and the solution space. We demonstrate the feasibility of some of the proposed components through a wide-area situational awareness experiment for the power grid across the three sites.

Network testbeds have become widely used in computer science, both for evaluation of research technologies and for hands-on teaching. This can naturally lead to oversubscription and resource allocation failures, as limited testbed resources cannot meet the increasing demand.

Researchers in the denial-of-service (DoS) field lack accurate, quantitative, and versatile metrics to measure service denial in simulation and testbed experiments. Without such metrics, it is impossible to measure severity of various attacks, quantify success of proposed defenses, and compare their performance. Existing DoS metrics equate service denial with slow communication, low throughput, high resource utilization, and high loss rate. These metrics are not versatile because they fail to monitor all traffic parameters that signal service degradation. They are not quantitative because they fail to specify exact ranges of parameter values that correspond to good or poor service quality. Finally, they are not accurate since they were not proven to correspond to human perception of service denial. We propose several DoS impact metrics that measure the quality of service experienced by users during an attack. Our metrics are quantitative: they map QoS requirements for several applications into measurable traffic parameters with acceptable, scientifically determined thresholds. They are versatile: they apply to a wide range of attack scenarios, which we demonstrate via testbed experiments and simulations. We also prove metrics' accuracy through testing with human users.

Testing security systems is challenging because a system's authors have to play the double role of attackers and defenders. Red team/blue team exercises are an invaluable mechanism for security testing. They partition researchers into two competing teams of attackers and defenders, enabling them to create challenging and realistic test scenarios. While such exercises provide valuable insight into vulnerabilities of security systems, they are very expensive and thus rarely performed. In this paper we describe a red team/blue team exercise, sponsored by DARPA's FTN program, and performed October 2002 --- May 2003. The goal of the exercise was to evaluate a collaborative DDoS defense, comprised of a distributed system, COSSACK, and a stand-alone defense, D-WARD. The role of the blue team was played by developers of the tested systems from USC/ISI and UCLA, the red team included researchers from Sandia National Laboratory, and all the coordination, experiment execution, result collection and analysis was performed by the white team from BBN Technologies. This exercise was of immense value to all involved --- it uncovered significant vulnerabilities in tested systems, pointed out desirable characteristics in DDoS defense systems (e.g., avoiding reliance on timing mechanisms), and taught us many lessons about testing of DDoS defenses.

Denial-of-service (DoS) research community lacks accurate metrics to evaluate an attack's impact on network services, its severity and the effectiveness of a potential defense. We propose several DoS impact metrics that measure the quality of service experienced by end users during an attack, and compare these measurements to application-specific thresholds. Our metrics are ideal for testbed experimentation, since necessary traffic parameters are extracted from packet traces gathered during an experiment.

Abstract — Denial of Service attacks have become a weapon for extortion and vandalism causing damages in the millions of dollars to commercial and government sites. Legal prosecution is a powerful deterrent, but requires attribution of attacks, currently a difficult task. In this paper we propose a method to automatically fingerprint and identify repeated attack scenarios—a combination of attacking hosts and attack tool. Such fingerprints not only aid in attribution for criminal and civil prosecution of attackers, but also help justify and focus response measures. Since packet contents can be easily manipulated, we base our fingerprints on the spectral characteristics of the attack stream which are hard to forge. We validate our methodology by applying it to real attacks captured at a regional ISP and comparing the outcome with header-based classification. Finally, we conduct controlled experiments to identify and isolate factors that affect the attack fingerprint. I.

Launching a denial of service (DoS) attack is trivial, but detection and response is a painfully slow and often a manual process. Automatic classification of attacks as single- or multi-source can help focus a response, but current packet-header-based approaches are susceptible to spoofing. This paper introduces a framework for classifying DoS attacks based on header content, transient ramp-up behavior and novel techniques such as spectral analysis. Although headers are easily forged, we show that characteristics of attack ramp-up and attack spectrum are more difficult to spoof. To evaluate our framework we monitored access links of a regional ISP detecting 80 live attacks. Header analysis identified the number of attackers in 67 attacks, while the remaining 13 attacks were classified based on ramp-up and spectral analysis. We validate our results through monitoring at a second site, controlled experiments, and simulation. We use experiments and simulation to understand the underlying reasons for the characteristics observed. In addition to helping understand attack dynamics, classification mechanisms such as ours are important for the development of realistic models of DoS traffic, can be packaged as an automated tool to aid in rapid response to attacks, and can also be used to estimate the level of DoS activity on the Internet. 1.