Large scale distributed systems generate so many metrics/features to monitor and analyze their performances. Most of the analysis to detect performance regressions and root causes of the regressions is done by manually looking at graphs of the metric values. This approach is error prone and doesn't scale. There are some recent works to automate root cause analysis. However these schemes either rely on specific network metrics or aggregate values such as median. Such approach, which doesn't use various physical and virtual device specific metrics, is ineffective in detecting anomalies and their root causes for large scale distributed systems/clusters such as VMware Virtual Storage Area Network (vSAN) and vSphere.

Various systems have natural groupings. For instance in large scale distributed system, we can have groups of virtual and/or physical devices. A system can also have groups of time series datasets collected at different time intervals. Such groups are usually characterized by multidimensional metrics (features) set. Clustering such groups using their multidimensional datasets has various applications, such as identifying different performance levels for anomaly detection and load balancing. Traditional algorithms focus on clustering a single time series dataset and not on such groups with multidimensional metrics datasets. In this paper, we present the design, implementation and analysis of two sets of group clustering algorithms. The first set is called one-to-many as it generates clusters of groups by comparing each group against all other groups. The second set of algorithms is called pairwise as it generates the clusters of groups using pairwise group dissimilarity matrix. Both sets of algorithms first generate group dissimilarity weights using metric ranking algorithms. We implemented the group clustering algorithms by extending a well known machine learning package and using a front-end visualizer.We validated the clustering algorithms using real world datasets on the VMware vSAN product. Experimental results show that 7 out of the 8 proposed algorithms can generate expected clusters in at least 4 out of the 6 detailed experiments. In 5 out of the 6 experiments, 3 out of the 8 proposed algorithms can generate the expected clusters. One of the pairwise algorithms can generate the expected clusters in all 6 of the 6 experiments.

With the fast growth of (online) content and the need for high quality content services, cloud data centers are increasingly becoming the preferred places to store data and retrieve it from. With a highly variable network traffic and limited resources, efficient server selection and data transfer rate allocation mechanisms become necessary. However, current approaches rely on random server selection schemes and inefficient data transmission rate control mechanisms. In this paper we present SCDA, an efficient server selection, resource allocation and enforcement mechanism with many salient features. SCDA has prioritized rate allocation mechanism to satisfy different service level agreements (SLA)s on throughput and delays. The allocation scheme can achieve max/min fairness. SCDA has a mechanism to detect and hence mitigate SLA violation in realtime. We have implemented SCDA in the NS2 simulator. Extensive experimental results confirm some of the design goals of SCDA to obtain a lower content transfer time and a higher throughput. The design of SCDA can achieve a content transfer time which is about 50% lower than the existing schemes and a throughput which is higher than existing approaches by upto than 60%.

Existing literature shows that Peer-to-Peer (P2P) content sharing can result in significant scalability gains in addition to assisting content distribution networks (CDNs). However, currently proposed CDN and P2P hybrid schemes do not provide accurate and efficient incentives to attract and maintain more peers. Besides, they do not use efficient prioritized congestion control and content source selection mechanisms to reduce content transfer time. We present Hincent, a quick content distribution protocol, which uses efficient prioritized rate allocation and content selection algorithms offering high incentives to participating peers. The fair incentives attract more peers which securely download and distribute contents. This in turn can benefit content providers and network operators. The Hincent rate allocations results in quicker content transfer time when compared with existing schemes. Hincent also employs effective rate enforcement mechanisms without requiring changes to the TCP/IP stack or to existing routers. Unlike existing centralized schemes such as YouTube, the design allows peers to have full control of (their) contents while sharing them with others using personal web servers. We have implemented Hincent in the NS2 simulator. Our detailed trace-based experiments show that Hincent outperforms existing schemes in terms of file download time and throughput by up to 30% on average. The results also demonstrate that Hincent obtains fair uplink prices for the uploaders and fair cost for the downloaders maintaining an overall system fairness. Besides, the results show the efficient Hincent enforcements of the prioritized allocations.

In this paper we analyze and study the impact of cloudlets in interactive mobile cloud applications. To study the impact we propose the design of cloudlet network and service architectures. Our study focuses on file editing, video streaming and collaborative chatting which are representative enterprise application scenarios. Initial simulation results show the performance gains of using cloudlets over using clouds in terms of data transfer delay and system throughput. When not more than two cloudlet wireless hops are used to transfer data, the cloudlet-based approach outperforms the cloud-based approach for all three application scenarios. With more cloudlet wireless hops under mobility, the cloud-based approach can give a better performance for some of the data transfers even though the cloudlet-based can outperform the cloud-based approach for most of the flows. In such scenarios, we suggest that an adaptive scheme should be used. For example, a scheme making an intelligent decision on either the cloudlet network or the cloud network, whichever gives minimum delay, can be used.

Applications like Twitter which use chat-like short messaging systems (SMS) have been widely used in public, political, military, emergency, humanitarian and other fields. Such applications usually involve servers (controllers) which control and forward messages from a sending client to a receiving client. The use of mobile wireless networks for such messaging systems has been increasing at a fast pace. To cope with this increase, there need to be efficient communication protocols and algorithms. To design such protocols and algorithms requires extensive analysis and understanding of the behavior of the communicating nodes under a given mobility scenario. One of the good metrics to understand the performance of such protocols is the reliability of message delivery. In this paper we present analytical models of the average reliability of short (chat-like) message delivery in mobile wireless networks as a multivariate function of the transmission range, movement area dimensions, number of servers (base stations) and message deadline (lifetime) under moderate realistic assumptions which can be easily relaxed and extended. Simulation results show that our analytical models give very good estimation of the average reliability of message delivery

This paper introduces the abstraction of energy reserves for sensor networks that virtualizes energy sources. It gives each of several applications sharing a platform the illusion of having its own private energy source. Energy virtualization is the next logical step in embedded systems after visualizing communication links and CPU capacity. Energy virtualization has not been addressed in past sensor network literature because most current wireless sensor networks feature single-user applications. To amortize deployment costs, future sensor networks, deployed in remote or hard- to-access areas, will likely be leveraged by scientists from different disciplines, each having their independent application for their individual research purposes. Platforms, planned for such deployment, will befitted with the union of sensors needed, but independent applications will share the remaining resources such as in-field storage and communication bandwidth, calling for quotas and isolation mechanisms. The most expensive resource shared in sensor networks is energy. This paper provides an energy isolation mechanism, called the virtual battery, that logically divides energy among applications to provide each its private energy reserve. An application can manage its private energy independently as if it were running alone on the platform. The application is terminated when its reserve is depleted. We implement and evaluate this abstraction on MicaZ motes running LiteOS. Our results show that the virtual battery mechanism succeeds at exporting the private reserve abstraction accurately and at a low overhead.

The transmission control protocol (TCP) is the dominant transport layer protocol for the end-to-end control of information transfer. Accurate models of TCP performance are a key and basic step for designing, dimensioning and planning IP (Internet Protocol) networks. Packet-level simulation models do not scale with the growth of network capacities and number of users. Measurements can only be used to explore existing network scenarios or otherwise they become costly and inflexible with the growth and complexity of the Internet. This study presents a simple, fast and accurate analytical model of TCP. The model gives Internet performance metrics, assuming that only basic network parameters such as the network topology, the number of users, link capacity, distance between network nodes and router buffer sizes are known. The TCP performance model derives performance metrics which express the network quality of service. To obtain the performance metrics, TCP and network sub-models are used. A closed network of ./G//spl infin/ queues is used to develop each TCP sub-model where each queue represents a state of a TCP connection. An M/M/1/K queue is used for each network sub-model which represents the output interface of an IP router with a buffer capacity of K packets. The two sub-models are iteratively solved. Initial comparisons of our results with ns2 simulation experiments and with results from a well known TCP model show that our model is faster, simpler and gives more accurate TCP performance metrics.

The feedback nature of reliable Internet protocols can be represented by a fixed point algorithm (FPA). The network signals congestion based on the aggregate traffic offered to it and each source adjusts its sending rate based on the feedback from the network to reach the network operating regime. The algorithms of reliable (transport) protocols are based on this fact. The FPA has been one of the most useful ways of solving analytical models of Internet performance. Apart from implementing the FPA for simple networks with few bottleneck links, no implementation algorithm of this complex procedure is given to the best of our knowledge. In this paper the author develops an implementation algorithm of the FPA for analytical models of the transmission control protocol (TCP) which is the most widely used transport protocol on the Internet and show how the FPA emulates TCP. Even though the algorithm considers TCP it can be easily generalized to any reliable Internet protocol. The implementation algorithm can be used to develop efficient and comprehensive analytical tools for the performance investigation of networks (the Internet) with an arbitrary number of bottleneck links and sources. Such tools help to analyze existing protocols and propose more efficient ones. In this paper the author also briefly discuss important ideas which can be used to prove the convergence of the FPA (and hence TCP) and the uniqueness of the fixed points. These ideas can be used to easily study the stability of the TCP protocol. Unlike previous proofs of convergence which apply to only specific network scenarios, our approach applies to different network scenarios