Modern cloud and data center platforms suffer failures and performance degradation from large traffic surges caused by both external (e.g., DDoS attacks) or internal (e.g., workload changes, operator errors, routing misconfigurations) factors. If not mitigated, traffic overload could have significant financial and availability implications for cloud providers. In this paper, we propose NetFuse, a mechanism to protect against traffic overload in OpenFlow-based data center networks. NetFuse is (1) scalable because it uses passively-collected OpenFlow control messages to detect active network flows; (2) accurate because it uses multi-dimensional flow aggregation to determine the right criteria to combine network flows that lead to overloading behavior; and (3) effective in limiting the damage of surges while not affecting the normal traffic because it uses a toxin-antitoxinlike mechanism to adaptively shape the rate of the flow based on application feedback. Experimental results on a real OpenFlow testbed show that NetFuse is effective in identifying and isolating misbehaving traffic with a small false positive rate (<; 9%).

Multi-tenant data centers are complex environments, running thousands of applications that compete for the same infrastructure resources and whose behavior is guided by (sometimes) divergent configurations. Small workload changes or simple operator tasks may yield unpredictable results and lead to expensive failures and performance degradation. In this paper, we propose a holistic approach for detecting operational problems in data centers. Our framework, FlowDiff, collects information from all entities involved in the operation of a data center -- applications, operators, and infrastructure -- and continually builds behavioral models for the operation. By comparing current models with pre-computed, known-to-be-stable models, FlowDiff is able to detect many operational problems, ranging from host and network failures to unauthorized access. FlowDiff also identifies common system operations (e.g., VM migration, software upgrades) to validate the behavior changes against planned operator tasks. We show that using passive measurements on control traffic from programmable switches to a centralized controller is sufficient to build strong behavior models; FlowDiff does not require active measurements or expensive server instrumentation. Our experimental results using NEC data center testbed, Amazon EC2, and simulations demonstrate that FlowDiff is effective and robust in detecting anomalous behavior. FlowDiff scales well with the number of applications running in the data center and their traffic volume.

With the ever-increasing number and complexity of applications deployed in data centers, the underlying network infrastructure can no longer sustain such a trend and exhibits several problems, such as resource fragmentation and low bisection bandwidth. In pursuit of a real-world applicable cloud network (CN) optimization approach that continuously maintains balanced network performance with high cost effectiveness, we design a topology independent resource allocation and optimization approach, NetDEO. Based on a swarm intelligence optimization model, NetDEO improves the scalability of the CN by relocating virtual machines (VMs) and matching resource demand and availability. NetDEO is capable of (1) incrementally optimizing an existing VM placement in a data center; (2) deriving optimal deployment plans for newly added VMs; and (3) providing hardware upgrade suggestions, and allowing the CN to evolve as the workload changes over time. We evaluate the performance of NetDEO using realistic workload traces and simulated large-scale CN under various topologies.

Driven by the large-scale growth of applications deployment in data centers and complicated interactions between service components, automated application dependency discovery becomes essential to daily system management and operation. In this paper, we present ADD, which extracts dependency paths for each application by decomposing the application-layer connectivity graph inferred from passive network monitoring data. ADD utilizes a series of statistical techniques and is based on the combination of global observation of application traffic matrix in the data center and local observation of traffic volumes at small time scales on each server. Compared to existing approaches, ADD is especially effective in the presence of overlapping and multi-hop applications and resilient to data loss and estimation errors.

With the ever-increasing number and complexity of applications deployed in data centers, the underlying network infrastructure can no longer sustain such a trend and exhibits several problems, such as resource fragmentation and low bisection bandwidth. In pursuit of a real-world applicable data center network (DCN) optimization approach that continuously maintains balanced network performance with high cost effectiveness, we design a topology independent resource allocation and optimization approach, NetDEO. Based on a swarm intelligence optimization model, NetDEO improves the scalability of the DCN by relocating virtual machines (VMs) and matching resource demand and availability. NetDEO is capable of (1) incrementally optimizing an existing VM placement in a data center; (2) deriving optimal deployment plans for newly added VMs; and (3) providing hardware upgrade suggestions and allowing the DCN to evolve as the workload changes over time. We evaluate the performance of NetDEO using realistic workload traces and simulated large-scale DCN under various topologies.

Network tomography has been proposed to ascertain internal network performances from end-to-end measurements. In this work, we present priority probing, an optimal probing scheme for unicast network delay tomography that is proven to provide the most accurate estimation. We first demonstrate that the Fisher information matrix in unicast network delay tomography can be decomposed into an additive form where each term can be obtained numerically. This establishes the space over which we can design the optimal probing scheme. Then, we formulate the optimal probing problem into a semi-definite programming (SDP) problem. High computation complexity constrains the SDP solution to only small scale scenarios. In response, we propose a greedy algorithm that approximates the optimal solution. Evaluations through simulation demonstrate that priority probing effectively increases estimation accuracy with a fixed number of probes.

In the recent years, a progressively growing number of computing and communication services have undertaken the migration from their conventional media to the new unified platform, IP networks. As a consequence, business success of service providers becomes largely determined by the effectiveness of their service management schemes, which require rapid identification of problems and resolution of network-related anomalies. However, this is a non-trivial task in heterogeneous enterprise networks due to service providers' invisibility of the health and performance of the underlying carrier network. In addition, the gap between quality of service (QoS) measurements reflecting network performance and quality of experience (QoE) metrics indicating user-perceived service quality further makes effective service management more challenging. In this paper, we present a unified service management system called QoEScope, which combines scalable end-to-end probing, accurate topology inference in the presence of implicit routers, adaptive bridging between QoS measurement and QoE metrics, and intelligent root cause analysis. Extensive testbed emulations and Internet experiments demonstrate that QoEScope is a highly practical and effective IP service management solution for heterogeneous enterprise networks.