When Google first introduced the Map/Reduce paradigm in 2004, no comparable system had been available to the general public. The situation has changed since then. The Map/Reduce paradigm has become increasingly popular and there is no shortage of Map/Reduce implementations in today's computing world. The predominant solution is currently Apache Hadoop, started by Yahoo. Besides employing custom Map/Reduce installations, customers of cloud services can now exploit ready-made made installations (e.g. the Elastic Map/Reduce System). In the mean time, other, second generation frameworks have started to appear. They either fine tune the Map/Reduce model for specific scenarios, or change the paradigm altogether, such as Google's Pregel. In this paper, we present a comparison between these second generation frameworks and the current de-facto standard Hadoop, by focusing on a specific scenario: large-scale graph analysis. We analyze the different means of fine-tuning those systems by exploiting their unique features. We base our analysis on the k-core decomposition problem, whose goal is to compute the centrality of each node in a given graph; we tested our implementation in a cluster of Amazon EC2 nodes with realistic datasets made publicly available by the SNAP project.

No abstract available.

It seems to be common wisdom that peer-to-peer overlay networks should reflect the structure of the underlying network to minimize the application-level latency, to increase the throughput, and to optimize the traffic for the Internet service providers. Network coordinates are one means to achieve this goal. However, previous real-world experiments were disappointing because methods like Vivaldi could not resolve the structure of the underlying network better than at a continental level. In our work, we have closely analyzed the reasons for that disappointing performance. Thereby, we were able to improve Vivaldi-like network coordinate systems so that they can now differentiate peers at a regional level. In this paper, we report on a large measurement study that we performed with the Vuze BitTorrent client. We show that these improved algorithms can indeed differentiate among different autonomous systems and different geographic regions. We also show that this achievement is less important than commonly thought and that the key contribution lies rather in the latency-optimal overlay formation itself.

Groups are both, a social phenomenon inherent to the human nature and a widely used structure in distributed systems. Recently, the Web 2.0 trend has begun to unite both aspects. Social networks provide their users with simple means to create, modify, join, and leave groups dynamically. Popular applications such as chat rooms and multi-player online games require social group structures. The same holds true for many e-science applications. But despite their growing importance, the structure of social groups has not yet been studied with respect to peer-to-peer or cloud computing systems. In this paper, we analyze the requirement structure that the different types of social groups induce in distributed systems. In particular, we focus on fully decentralized peer-to-peer systems. First, we formalize the requirements using results from field studies in social networks. Then we classify the various group management models by considering their differences in group creation and management. We also discuss scalability, maintainability, security issues and privacy guarantees of the different group models.

Network coordinates have become an important building block for efficient peer-to-peer applications. They allow the peers to determine those other peers to whom they are likely to experience a low round trip time. So they can preferably attach to these selected peers and thereby create efficient overlay structures. Despite the substantial interest in this topic, there is still room for improvement. In this paper we propose Hierarchical Vivaldi, a variant of the popular Vivaldi algorithm. It predicts the peers' embedding error so that it can optimize the peer selection process. We conducted extensive studies with several sets of Internet traffic traces. They compare our proposal to Vivaldi and its previously proposed optimizations. Our results show that Hierarchical Vivaldi can improve the accuracy of the RTT predictions by an order of magnitude.

IgorFs is a distributed, decentralized peer-to-peer (P2P) file system that is completely transparent to the user. It is built on top of the Igor peer-to-peer overlay network, which is similar to Chord, but provides additional features like service orientation or proximity neighbor and route selection. IgorFs offers an efficient means to publish data files that are subject to frequent but minor modifications. In our demonstration we show two use cases for IgorFs: the first example is (static) software-distribution and the second example is (dynamic) file distribution.