Traditionally, performance analysis, de-bugging, triaging, troubleshooting, and optimization are left in the hands of performance experts. The main rationale behind this is that performance engi-neering is considered a specialized do-main expertise, and therefore left to the trained hands of experts. However, this approach requires human manpower to be put behind every performance escala-tion. This is no longer future proof in enterprise environments because of the following reasons: (i) Enterprise customers now expect much quicker performance troubleshooting, particularly in cloud platforms as Soft-ware As A Service (SaaS) offerings where the billing is subscription based, (ii) As products grow more distributed and complex, the number of performance met-rics required to troubleshoot a perfor-mance problem implodes, making it very time consuming for human intervention and analysis, and (iii) Our past experi-ences show that while many customers land up on similar performance issues, the human effort to troubleshoot each of these performance issues in a different infrastructural environment is non-trivial. We believe that data analytics platforms that can quickly mine through performance data and point out potential bottlenecks offer a good solution for non-domain experts to debug and solve a performance issue. In this work, we showcase a cloud based performance data analytics framework which can be leveraged to build tools which analyze and root-cause performance issues in enterprise sys-tems. We describe the architecture of this framework which consists of: (i) A cloud service (which we term as a plugin), (ii) Supporting libraries that may be used to interact with this plugin from end-systems such as computer serv-ers or appliance Virtual Machines (VMs), and (iii) A solution to monitor and ana-lyze the results delivered by the plugin. We demonstrate how this platform can be used to develop different perfor-mance analyses and debugging tools. We provide one example of a tool that we have built on top of this framework and released: VMware Virtual SAN (vSAN) per-formance diagnostics.

Large scale distributed systems generate so many metrics/features to monitor and analyze their performances. Most of the analysis to detect performance regressions and root causes of the regressions is done by manually looking at graphs of the metric values. This approach is error prone and doesn't scale. There are some recent works to automate root cause analysis. However these schemes either rely on specific network metrics or aggregate values such as median. Such approach, which doesn't use various physical and virtual device specific metrics, is ineffective in detecting anomalies and their root causes for large scale distributed systems/clusters such as VMware Virtual Storage Area Network (vSAN) and vSphere.

Various systems have natural groupings. For instance in large scale distributed system, we can have groups of virtual and/or physical devices. A system can also have groups of time series datasets collected at different time intervals. Such groups are usually characterized by multidimensional metrics (features) set. Clustering such groups using their multidimensional datasets has various applications, such as identifying different performance levels for anomaly detection and load balancing. Traditional algorithms focus on clustering a single time series dataset and not on such groups with multidimensional metrics datasets. In this paper, we present the design, implementation and analysis of two sets of group clustering algorithms. The first set is called one-to-many as it generates clusters of groups by comparing each group against all other groups. The second set of algorithms is called pairwise as it generates the clusters of groups using pairwise group dissimilarity matrix. Both sets of algorithms first generate group dissimilarity weights using metric ranking algorithms. We implemented the group clustering algorithms by extending a well known machine learning package and using a front-end visualizer.We validated the clustering algorithms using real world datasets on the VMware vSAN product. Experimental results show that 7 out of the 8 proposed algorithms can generate expected clusters in at least 4 out of the 6 detailed experiments. In 5 out of the 6 experiments, 3 out of the 8 proposed algorithms can generate the expected clusters. One of the pairwise algorithms can generate the expected clusters in all 6 of the 6 experiments.

The transmission capacity of today's high-speed networks is often greater than the capacity of an end-system (such as a server or a remote client) to consume the incoming data. The mismatch between the network and the end-system, which can be exacerbated by high end-system workloads, will result in incoming packets being dropped at different points in the packet receiving process. In particular, a packet may be dropped in the NIC, in the kernel ring buffer, and (for rate based protocols) in the socket buffer. To provide reliable data transfers, these losses require retransmissions, and if the loss rate is high enough result in longer download times. In this paper, we focus on UDP-like rate based transport protocols, and address the question of how best to estimate the rate at which the end-system can consume data which minimizes the overall transfer time of a file.

In many e-science applications, there exists an important need to aggregate information from data repositories distributed around the world. In an effort to better link these resources in a unified manner, many lambda-grid networks, which provide end-to-end dedicated optical-circuit-switched connections, have been investigated. In this context, we consider the problem of aggregating files from distributed databases at a (grid) computing node over a lambda grid. The challenge is (1) to identify routes (that is, circuits) in the lambda-grid network, along which files should be transmitted, and (2) to schedule the transfers of these files over their respective circuits. To address this challenge, we propose a hybrid approach that combines offline and online scheduling. We define the Time-Path Scheduling Problem (TPSP) for offline scheduling. We prove that TPSP is NP-complete, develop a Mixed Integer Linear Program (MILP) formulation for TPSP, and then propose a greedy approach to solve TPSP because the MILP does not scale well. We compare the performance of the greedy approach on a few representative lambda-grid network topologies. One key input to the offline schedule is the file transfer time. Due to dynamics at the receiving end host, which is hard to model precisely, the actual file transfer time may vary. We first propose a model for estimating the file transfer time. Then, we propose online reconfiguration algorithms so that as files are transferred, the offline schedule may be modified online, depending on the amount of time that it actually took to transfer the file. This helps in reducing the total time to transfer all the files, which is an important metric. To demonstrate the effectiveness of our approach, we present results on an emulated lambda-grid network testbed.

With the development of optical technology, the span of a broadband access network using passive optical network (PON) technology can be increased from today's standard of 20 km to 100 km or higher. As a result, we have the long-reach (LR) PON which not only has extended reach, but which can also support a large base of users by employing wavelength-division multiplexing (WDM) for its data transmissions. However, a major challenge in the LR-PON is that the propagation delay (for data as well as control signals) between the telecom central office (CO) and the end user is increased by a very significant amount. Now, traditional PON algorithms for scheduling the upstream transmission, such as dynamic bandwidth allocation (DBA) algorithms, may not be sufficient; actually, they may lead to degraded performance because of the long delay of the "control loop" between the CO and the users. This challenge motivates us to investigate a multi-thread polling algorithm to distribute the upstream bandwidth dynamically in the LR-PON. In this study, we analyze key parameters of the algorithm, such as initiating and tuning multiple threads. We then demonstrate the algorithm's advantage to decrease the average packet delay under varying offered loads.

A control-plane architecture for supporting advance reservation of dedicated bandwidth channels on a switched network infrastructure is described including the front-end web interface, user and token management scheme, bandwidth scheduler, and signaling daemon. A path computation algorithm for bandwidth scheduling is proposed based on an extension of Bellman-Ford algorithm to an algebraic structure on sequences of disjoint non-negative real intervals. An implementation of this architecture for UltraScience Net is briefly described.

Next-generation e-science applications will require the ability to transfer information at high data rates between distributed computing centers and data repositories. To support such applications, lambda grid networks have been built to provide large, on-demand bandwidth between end-points that are interconnected via optical circuit-switched lambdas. It is extremely important to develop an efficient transport protocol over such high-capacity, dedicated circuits. Because lambdas provide dedicated bandwidth between endpoints, they obviate the need for network congestion control. Consequently, past research has demonstrated that rate-based transport protocols, such as RBUDP, are more effective than TCP in transferring data over lambdas. However, while lambdas eliminate congestion in the network, they ultimately push the congestion to the endpoints - congestion that current rate-based transport protocols are ill-suited to handle. In this paper we introduce a "rate-adaptive protocol for intelligent delivery (RAPID)" of data that is lightweight and end-system performance-aware, so as to maximize end-to-end throughput while minimizing packet loss. Based on self monitoring of the dynamic task-priority at the receiving end-system, our protocol enables the receiver to proactively deliver feedback to the sender, so that the sender may adapt its sending rate to avoid congestion at the receiving end-system. This avoids large bursts of packet losses typically observed in current rate-based transport protocols. Over a 10-Gigabit link emulation of an optical circuit, RAPID reduces file-transfer time, and hence improves end-to-end throughput by as much as 25%.

The Passive Optical Network (PON) is an attractive solution for high-bandwidth access networks. In the context of a broadband access network, the term open access implies the ability of multiple service providers to share the deployed access network infrastructure to make services available to the end users. Multiple services may thereby be delivered over a shared access channel. Open access requires fairness in terms of throughput, delay, jitter, and other network parameters in the access channel among the sharing entities, namely service providers and end users. Since the traffic in an access network is very bursty, an access network may be frequently subjected to high loads for certain durations of time. Meeting the above fairness requirements under such conditions is therefore very challenging. In this study, we first motivate the problem of meeting fairness requirements simultaneously to both service providers and users, which are located at opposite ends of an access channel. We then investigate the importance of two different sets of Service-Level Agreements (SLAs), which we call Dual SLAs. After formulating a mathematical model, we propose an efficient scheduling algorithm to meet Dual SLAs which is based on the well-known concept of max-min fairness. We then demonstrate the effectiveness of our proposed algorithm through simulations using a discrete-event-simulator-based PON set-up, which compares the fairness of the Dual-SLA scheduling algorithm with that of other traditional fair queuing algorithms such as Deficit Round Robin (DRR)

Ethernet passive optical networks (EPONs) are an attractive solution for meeting the broadband access requirements of residential users. Open access is a regulatory requirement in many countries which mandates that the access infrastructure must be open to various service providers for free competition. Thus, different service providers may cater services to same or different users using the same shared access channel. Open access demands fairness in the use of network resources among sharing entities, namely service providers and end users. In this study, we investigate scheduling algorithms for fair bandwidth sharing in the context of open access for EPONs.

The problem of aggregating large data files from distributed databases and address the corresponding challenges involved from a network architecture perspective is considered. We model this problem as one of identifying a time-path schedule (TPS) in a graph representation of the network. We prove that the TPS problem (TPSP) is NP-complete. We then propose a mixed integer linear programming (MILP)-based approach and three heuristics longest-file-first (LFF), disjoint-paths (DP), and most-distant-file-first (MDFF) - to solve TPSP.

In this paper, we propose and investigate the characteristics of a fair queueing with service envelopes (FQSE) algorithm-a hierarchical fair-share scheduling algorithm for access networks based on a remote scheduling system such as Ethernet passive optical networks (EPON) or cable TV network. FQSE is designed to overcome the limiting factors of a typical remote scheduling system such as large control-plane delay, limited control-plane bandwidth, and significant queue switch-over overhead. The algorithm is based on a concept of service envelope-a function representing the fair allocation of resources based on a global network condition called satisfiability parameter (SP). We define properties of cousin-fairness and sibling-fairness and show the FQSE to be cousin-fair. FQSE is unique in that it is the only hierarchical algorithm that is simultaneously cousin-fair. Furthermore, we show the necessary techniques to adapt FQSE to variable-sized packet-based networks. We analyze FQSE performance in EPON serving 1024 independent queues and demonstrate FQSE's ability to provide guaranteed bandwidth to each queue and to share the excess bandwidth fairly.