The performance of distributed applications often critically depends on the throughput of the interconnecting network: how fast data can be carried across a network. Over the last years, great progress has been made in understanding demand-oblivious throughput: how fast a given demand matrix describing pairwise communication requirements, can be served on a given network. However, surprisingly little is known today about the achievable demand-aware throughput: the throughput on a network topology which can be optimized toward the demand. Such demand-aware networks have recently gained popularity in datacenters and are enabled by emerging reconfigurable optical technologies.

Small-world networks are attractive for the efficient routing they provide, requiring only a low link density. They have hence also been considered for the design of distributed systems, such as peer-to-peer networks. However, existing small-world network designs are oblivious to the actual traffic they serve. In this paper, we initiate the study of demand-aware small-world networks. In particular, we extend the Kleinberg graph model, by allowing the nodes to choose the distribution of long-range links according to the traffic demand. We present a formal analysis of the weighted route lengths for the important case of clustered demands. We show both in theory and in simulations, using real-world traffic workloads, that demand-aware small-world graphs can significantly outperform their demand-oblivious counterparts.

In-network computing via smart networking devices is a recent trend in modern datacenter networks. State-of-the-art switches with near line-rate computing and aggregation capabilities enable acceleration and improved resource utilization for modern applications like large-scale distributed and federated machine learning, as well as big data analytics. We study the problem of activating a limited number of in-network computing devices within a network, aiming at reducing the overall cost incurred by such a deployment. Such limitations on the number of in-network computing elements arise, e.g., in incremental upgrades of network infrastructure, and are also due to requiring specialized middleboxes, or FPGAs, for supporting heterogeneous workloads, and multiple tenants. We present an efficient optimal algorithm for placing such devices in tree networks with arbitrary link rates, and further evaluate its performance in various scenarios and for various tasks, including federated/distributed ML and big data analytics. Our results show that even a small fraction of network devices supporting in-network aggregation leads to a significant reduction in network utilization cost. Furthermore, we show that various intuitive strategies for performing such placements are significantly inferior compared with our solution, for varying workloads, tasks, and link rates.

While the traditional design of network topologies is demand oblivious, recent advances in reconfigurable networks enable real-time and dynamic communication network topologies, e.g., in datacenter networks. This trend motivates a new paradigm where topologies can adjust to the demand they need to serve. We consider the static and distributed version of this network design problem where the input is a request distribution, $\mathcal{D}$ (demand matrix), and a bound Δ, on the maximum degree of the output topology. In turn, the objective is to design an (undirected) demand-aware network N of bounded-degree Δ, which minimizes the expected path length (with respect to $\mathcal{D}$).This paper draws a connection between the k-root of graphs and the network design problem and uses forest-decomposition of the demand matrix as the primary methodology. In turn, we provide new algorithms for demand-aware network design, including cases where our algorithms are (order) optimal and improve previous results. In addition, we provide, for the first time and for the case of bounded arboricity, (i) an efficient distributed algorithm for the CONGEST model and (ii) an efficient and PRAM-based parallel algorithm. We also present empirical results on real-world demand matrices where our algorithms produce both low-degree and low-expected path length network designs.

We consider the fundamental problem of designing a self-adjusting tree, which efficiently and locally adapts itself towards the demand it serves (namely accesses to the items stored by the tree nodes), striking a balance between the benefits of such adjustments (enabling faster access) and their costs (reconfigurations). This problem finds applications, among others, in the context of emerging demand-aware and reconfigurable datacenter networks and features connections to self-adjusting data structures. Our main contribution is SeedTree, a dynamically optimal self-adjusting tree which supports local (i.e., greedy) routing, which is particularly attractive under highly dynamic demands. SeedTree relies on an innovative approach which defines a set of unique paths based on randomized item addresses, and uses a small constant number of items per node. We complement our analytical results by showing the benefits of SeedTree empirically, evaluating it on various synthetic and real-world communication traces.

The performance of many data-centric cloud applications critically depends on the performance of the underlying datacenter network. Reconfigurable optical technologies have recently introduced a novel opportunity to improve datacenter network performance, by allowing to dynamically adjust the network topology according to the demand. However, the vision of self-adjusting networks raises the fundamental question how such networks can be efficiently operated in a scalable and distributed manner. This article presents$DiSplayNet$, the first fully distributed self-adjusting network.$DiSplayNet$relies on algorithms that perform decentralized and concurrent topological adjustments to account for changes in the demand. We propose two natural metrics to evaluate the performance of distributed self-adjusting networks, theamortized work(the cost of routing on and adjusting the network) and themakespan(the time it takes to serve a set of communication requests). We present a rigorous formal analysis of the work and makespan of$DiSplayNet$, which can be seen as an interesting generalization of analyses known from sequential self-adjusting datastructures. We complement our theoretical contribution with an extensive trace-driven simulation study, shedding light on the opportunities and limitations of leveraging spatial and temporal locality and concurrency in self-adjusting networks.

We revisit the design of self-adjusting single-source tree networks. The problem can be seen as a generalization of the classic list update problem to trees, and finds applications in reconfigurable datacenter networks. We are given a balanced binary tree T connecting n nodes V = {v1,…, vn}. A source node v0, attached to the root of the tree, issues communication requests to nodes in V , in an online and adversarial manner; the access cost of a request to a node v, is given by the current depth of v in T . The online algorithm can try to reduce the access cost by performing swap operations, with which the position of a node is exchanged with the position of its parent in the tree; a swap operation costs one unit. The objective is to design an online algorithm which minimizes the total access cost plus adjustment cost (swapping). Avin et al. [12] (LATIN 2020) recently presented RANDOM-PUSH, a constant competitive online algorithm for this problem, based on random walks, together with a sophisticated analysis exploiting the working set property.This paper studies analytically and empirically, online algorithms for this problem. In particular, we explore how to derandomize RANDOM-PUSH. In the analytical part, we consider a simple derandomized algorithm which we call ROTOR-PUSH, as its behavior is reminiscent of rotor walks. Our first contribution is a proof that ROTOR-PUSH is constant competitive: its competitive ratio is 12 and hence by a factor of five lower than the best existing competitive ratio. Interestingly, in contrast to RANDOM-PUSH, the algorithm does not feature the working set property, which requires a new analysis. We further present a significantly improved and simpler analysis for the randomized algorithm, showing that it is 16-competitive.In the empirical part, we compare all self-adjusting single-source tree networks, using both synthetic and real data. In particular, we shed light on the extent to which these self-adjusting trees can exploit temporal and spatial structure in the workload. Our experimental artefacts and source codes are publicly available.

Distributed computing has become a common practice nowadays, where recent focus has been given to the usage of smart networking devices with in-network computing capabilities. State-of-the-art switches with near-line rate computing and aggregation capabilities enable acceleration and improved performance for various modern applications like big data analytics and large-scale distributed and federated machine learning.In this work, we formulate and study the theoretical algorithmic foundations of such approaches, and focus on how to deploy and use constrained in-network computing capabilities within the data center. We focus our attention on reducing the network congestion, i.e., the most congested link in the network, while supporting the given workload(s). We present an efficient optimal algorithm for tree-like network topologies and show that our solution provides as much as an x13 improvement over common alternative approaches. In particular, our results show that having merely a small fraction of network devices that support in-network aggregation can significantly reduce the network congestion, both for single and multiple workloads.

Emerging communication technologies allow to reconfigure the physical network topology at runtime, enablingdemand-aware networks (DANs): networks whose topology is optimized toward the workload they serve. However, today, only little is known about the fundamental algorithmic problems underlying the design of such demand-aware networks. This paper presents the first bounded-degree, demand-aware network,$\textit {cl-DAN} $, which minimizesbothcongestion and route lengths. The degree bound$\Delta $is given as part of the input. The designed network is provably (asymptotically) optimal in each dimension individually: we show that there do not exist any bounded-degree networks providing shorter routes (independently of the load), nor do there exist networks providing lower loads (independently of the route lengths). The main building block of the designed$\textit {cl-DAN} $networks are$\textit {ego-trees}$: communication sources arrange their communication partners in an optimal tree,individually. While the union of these ego-trees forms the basic structure of$\textit {cl-DANs}$, further techniques are presented to ensure bounded degrees (for scalability).

This paper studies a fundamental algorithmic problem related to the design of demand-aware networks: networks whose topologies adjust toward the traffic patterns they serve, in an online manner. The goal is to strike a tradeoff between the benefits of such adjustments (shorter routes) and their costs (reconfigurations). In particular, we consider the problem of designing a self-adjusting tree network which serves single-source, multi-destination communication. The problem is a central building block for more general self-adjusting network designs and has interesting connections to self-adjusting datastructures. We present two constant-competitive online algorithms for this problem, one randomized and one deterministic. Our approach is based on a natural notion of Most Recently Used (MRU) tree, maintaining a working set. We prove that the working set is a cost lower bound for any online algorithm, and then present a randomized algorithm RANDOM- PUSH which approximates such an MRU tree at low cost, by pushing less recently used communication partners down the tree, along a random walk. Our deterministic algorithm Move-Half does not directly maintain an MRU tree, but its cost is still proportional to the cost of an MRU tree, and also matches the working set lower bound.

In order to meet the increasingly stringent throughput and latency requirements in datacenter networks, several innovative network architectures based on reconfigurable optical topologies have been proposed. Examples include demand-oblivious reconfigurable topologies such as RotorNet (SIGCOMM 2017), Opera (NSDI 2020), and Sirius (SIGCOMM 2021), as well as demand-aware topologies such as ProjecToR (SIGCOMM 2016). All these architectures feature attractive performance properties using specific prototypes. However, reproducing these experiments is often difficult due to missing hardware and publicly available software. This paper presents a flexible framework for reconfigurable networks based on off-the-shelf hardware, which supports experimentation and reproducibility at a small scale. We describe how our framework, ExReC, can be instantiated with different configurations, allowing us to emulate existing architectures and to study their trade-offs. Finally, we demonstrate the application of our approach to different use cases and workloads, including distributed machine learning training.

In-network computing via smart networking devices is a recent trend for modern datacenter networks. State-of-the-art switches with near line rate computing and aggregation capabilities are developed to enable, e.g., acceleration and better utilization for modern applications like big data analytics, and large scale distributed and federated machine learning. We formulate and study the problem of activating a limited number of in-network computing devices within a network, aiming at reducing the overall network utilization for a given workload. Such limitations on the number of in-network computing elements per workload arise, e.g., in incremental upgrades of network infrastructure, and are also due to requiring specialized middleboxes, or FPGAs, that should support heterogeneous workloads, and multiple tenants.

Emerging optical communication technologies support the dynamic reconfiguration of datacenter network topologies depending on the traffic they serve. However, to reap the benefits of such demand-aware networks, a control logic is required which allows to quickly learn and adapt to traffic patterns. This paper presents CacheNet, a novel approach to efficiently control demand-aware networks. CacheNet leverages temporal and spatial locality in the traffic by managing the reconfigurable links of the optical switches as a links-cache. Network traffic, in turn, can be served either by a link from the link-cache component or by a demand-oblivious topology component. We study several classic caching algorithms and provide an analytical model which captures their performance benefits compared to an all demand-oblivious topology. Our analytical results show that based on the hit ratios and the links-cache size, our hybrid design can outperform designs that are based only on demand-oblivious topology.

This paper explores the design of dynamic network topologies which adjust to the workload they serve, in a demand-aware and online manner. Such self-adjusting networks (SANs) are enabled by emerging optical technologies, and can be found, e.g., in datacenters. SANs can be used to reduce routing costs by moving frequently communicating nodes topologically closer. However, such reconfigurations also come at a cost, introducing a need for online algorithms which strike an optimal balance between the benefits and costs of reconfigurations.This paper presents SANs which provide, for the first time, provable working set guarantees: the routing cost between node pairs is proportional to how recently these nodes communicated last time. Our SANs rely on a distributed implementation of skip lists (which serves as the topology) and provide additional interesting properties such as local routing. Our first contribution is SASL2, which is a randomized and sequential SAN algorithm that achieves the working set property. Then we show how SASL2can be converted to a distributed algorithm that handles concurrent communication requests and maintains SASL2's properties. Finally, we present deterministic SAN algorithms.

This paper initiates the study of the classic balanced graph partitioning problem from an online perspective: Given an arbitrary sequence of pairwise communication requests between $n$ nodes, with patterns that may change over time, the objective is to service these requests efficiently by partitioning the nodes into $L$ clusters, each of size $k$, such that frequently communicating nodes are located in the same cluster. The partitioning can be updated dynamically by migrating nodes between clusters. The goal is to devise online algorithms which jointly minimize the amount of intercluster communication and migration cost. The problem features interesting connections to other well-known online problems. For example, scenarios with $L = 2$ generalize online paging, and scenarios with $k = 2$ constitute a novel online variant of maximum matching. We present several lower bounds and algorithms for settings both with and without cluster-size augmentation. In particular, we prove that any deterministic online algorithm has a competitive ratio of at least $k$, even with significant augmentation. Our main algorithmic contributions are an $O(k \log k)$-competitive deterministic algorithm for the general setting with constant augmentation and a constant competitive algorithm for the maximum matching variant.

In the future, analysis of social networks will conceivably move from graphs to hypergraphs. However, theory has not yet caught up with this type of data organizational structure. By introducing and analyzing a general model of preferential attachment hypergraphs, this paper makes a step towards narrowing this gap. We consider a random preferential attachment model H(p, Y) for network evolution that allows arrivals of both nodes and hyperedges of random size. At each time step t, two possible events may occur: (1) [vertex arrival event:] with probability p > 0 a new vertex arrives and a new hyperedge of size Yt, containing the new vertex and Yt − 1 existing vertices, is added to the hypergraph; or (2) [hyperedge arrival event:] with probability 1 − p, a new hyperedge of size Yt, containing Yt existing vertices, is added to the hypergraph. In both cases, the involved existing vertices are chosen independently at random according to the preferential attachment rule, i.e., with probability proportional to their degree, where the degree of a vertex is the number of edges containing it. Assuming general restrictions on the distribution of Yt, we prove that the H(p, Y) model generates power law networks, i.e., the expected fraction of nodes with degree k is proportional to k−1−⌈, where [EQUATION]. This extends the special case of preferential attachment graphs, where Yt = 2 for every t, yielding ⌈ = 2/(2 − p). Therefore, our results show that the exponent of the degree distribution is sensitive to whether one considers the structure of a social network to be a hypergraph or a graph. We discuss, and provide examples for, the implications of these considerations.

Emerging communication technologies allow to reconfigure the physical network topology at runtime, enabling demand-aware networks (DANs): networks whose topology is optimized toward the workload they serve. However, today, only little is known about the fundamental algorithmic problems underlying the design of such demand-aware networks. This paper presents the first bounded-degree, demand-aware network, ct-DAN, which minimizes both congestion and route lengths. The designed network is provably (asymptotically) optimal in each dimension individually: we show that there do not exist any bounded-degree networks providing shorter routes (independently of the load), nor do there exist networks providing lower loads (independently of the route lengths). The main building block of the designed ct-DAN networks are ego-trees: communication sources arrange their communication partners in an optimal tree, individually. While the union of these ego-trees forms the basic structure of cl-DANs, further techniques are presented to ensure bounded degrees (for scalability).

We consider the problem of designing dynamic network topologies that self-adjust to the (possibly changing) traffic pattern they serve. Such demand-aware networks currently receive much attention, especially in the context of datacenters, due to emerging technologies supporting the fast reconfiguration of the physical topology. We present the first fully distributed, provably efficient self-adjusting network. Our network called DiSptayNet relies on algorithms that perform decentralized and concurrent topological adjustments to account for changes in the demand. We present a rigorous formal analysis of the correctness and performance of DiSptayNet, which can be seen as an interesting generalization of analyses known from sequential self-adjusting datastructures. We also report on results from extensive trace-driven simulations.

In order to support emerging data-intensive applications, many clever frameworks have been developed over the last years to efficiently and distributedly process big data sets, such as MapReduce. However, these frameworks are often optimized for relatively homogeneous environments, and accounting, e.g., for the varying connectivity of wide-area network infrastructure, may require complex placement algorithms. In this paper, we present Nap, which allows optimizing distributed data processing frameworks such as MapReduce for heterogeneous environments. Nap allows adapting resources dynamically, without requiring complex placement or migration algorithms, or modifications to the logic of the mappers and reducers. Rather, Nap simply changes the data partition, by spawning virtual nodes (e.g., reducers) depending on the demand. To this end, Nap leverages a connection to integer partition problems and employs Young lattices to guarantee minimal completion times (i.e., the makespan). In fact, Nap comes with provable performance guarantees and also supports applications that leverage redundancy to speed up executions further. In particular, to demonstrate our framework, as a case study, we show how to execute multiway joins across wide-area networks with limited bandwidth efficiently. Our experiments, based on a proof-of-concept prototype implementation, confirm the potential of Nap to reduce completion times.

This paper explores the design of dynamic network topologies which adjust to the workload they serve, in an online manner. Such self-adjusting networks (SANs) are enabled by emerging optical technologies, and can be found, e.g., in datacenters. SANs can be used to reduce routing costs by moving frequently communicating nodes topologically closer. This paper presents SANs which provide, for the first time, provable working set guarantees: the routing cost between node pairs is proportional to how recently these nodes communicated last time. Our SANs rely on skip lists (which serve as the topology) and provide additional interesting properties such as local routing.

It is well known that individuals in social networks tend to exhibit homophily, the preference of people to associate with others from the same social group or type. Graph assortativity or Modularity is the most accepted measure for the homophily level of the whole network. It is well defined for simple networks where each node has a single type, and edges are unweighted. In this work, we extend modularity and assortativity in several ways. First, we define type assortativity which measures the homophily level of each type and enable the comparison between types of different size within the network. Second, we extend the measures to the case of nodes with multiple types and weighted edges. We evaluate our definitions on a weighted, research collaboration, social network between the most cited authors in the ACM digital library. We use nationality-based multiple types where a author can belong to multiple nationalities. While nationality-based homophily is trivial when the network is large (based on local research at universities) our empirical results show that even for the top 1000 authors a high level of nationality-based homophily exists, and different nationalities exhibit a different level of homophily.

This paper demonstrates that the Preferential Attachment rule naturally emerges in the context of evolutionary network formation, as the unique Nash equilibrium of a simple social network game. In this game, each node aims at maximizing its degree in the future, representing its social capital in the "society" formed by the nodes and their connections. This result provides additional formal support to the commonly used Preferential Attachment model, initially designed to capture the "rich get richer" aphorism. In the process of establishing our result, we expose new connections between Preferential Attachment, random walks, and Young»s Lattice.

Consider a random preferential attachment model G(p) for network evolution that allows both node and edge arrivals. Starting with an arbitrary nonempty graph G0, at each time step, there are two possible events: with probability p > 0 a new node arrives and a new edge is added between the new node and an existing node, and with probability 1 - p a new edge is added between two existing nodes. In both cases, the involved existing nodes are chosen at random according to preferential attachment, i.e., with probability proportional to their degree. G(p) is known to generate power law networks, i.e., the fraction of nodes with degree k is proportional to k-β. Here β=(4-p)/(2-p) is in the range (2,3].

Traditionally, networks such as datacenter interconnects are designed to optimize worst-case performance under arbitrary traffic patterns. Such network designs can however be far from optimal when considering the actual workloads and traffic patterns which they serve. This insight led to the development of demand-aware datacenter interconnects which can be reconfigured depending on the workload. Motivated by these trends, this paper initiates the algorithmic study of demand-aware networks (DANs), and in particular the design of bounded-degree networks. The inputs to the network design problem are a discrete communication request distribution, D, defined over communicating pairs from the node set V, and a bound, d, on the maximum degree. In turn, our objective is to design an (undirected) demand-aware network N = (V,E) of bounded-degree d, which provides short routing paths between frequently communicating nodes distributed across N. In particular, the designed network should minimize the expected path length on N (with respect to D), which is a basic measure of the efficiency of the network. We show that this fundamental network design problem exhibits interesting connections to several classic combinatorial problems and to information theory. We derive a general lower bound based on the entropy of the communication pattern D, and present asymptotically optimal network-aware design algorithms for important distribution families, such as sparse distributions and distributions of locally bounded doubling dimensions.

SplayNets are reconfigurable networks which adjust to the communication pattern over time. We present DiSplayNets, a distributed (concurrent and decentralized) implementation of SplayNets.

This paper initiates the study of locally self-adjusting networks: networks whose topology adapts dynamically and in a decentralized manner, to the communication pattern σ. Our vision can be seen as a distributed generalization of the self-adjusting datastructures introduced by Sleator and Tarjan, 1985: In contrast to their splay trees which dynamically optimize the lookup costs from a single node (namely the tree root), we seek to minimize the routing cost between arbitrary communication pairs in the network. As a first step, we study distributed binary search trees (BSTs), which are attractive for their support of greedy routing. We introduce a simple model which captures the fundamental tradeoff between the benefits and costs of self-adjusting networks. We present the SplayNet algorithm and formally analyze its performance, and prove its optimality in specific case studies. We also introduce lower bound techniques based on interval cuts and edge expansion, to study the limitations of any demand-optimized network. Finally, we extend our study to multi-tree networks, and highlight an intriguing difference between classic and distributed splay trees.

Is there a bias in paper selection processes for conferences? This work addresses one aspect of this question, and empirically examines if there is a bias in favor of the collaborators of the technical program committee members. Specifically, we check whether a paper written by a past collaborator of a program committee member is more likely to be accepted to the conference. If so, one might say that the program committee members were biased; if not, then they are fair. In order to answer the bias question, we studied 12 ACM/IEEE conferences over several years. For each annual meeting of a conference we constructed its social network, whose vertices are the program committee members and the authors of the papers accepted to the meeting. Two researchers are collaborators (neighbors in the network) if they have co-authored a paper before the meeting. In turn, for each meeting network, we calculated the coverage of the program committee in the network, which is the ratio between the number of authors that are collaborators of the program committee, and the total number of the authors-vertices of the meeting. We compared the coverage of the real meeting's social networks, to the coverage in artificially generated meetings (random and others). We view a program committee as coverage biased if its coverage is significantly higher than that of corresponding artificially generated meetings of the conference. Our findings show that, although there are some coverage biased program committees, in most meetings, the coverage in the real meetings is the same as, and sometimes less than, the artificially generated ones, indicating that on average there is probably no bias in favor of papers written by collaborators of the program committee members for these high quality conferences.

The glass ceiling effect has been defined in a recent US Federal Commission report as "the unseen, yet unbreakable barrier that keeps minorities and women from rising to the upper rungs of the corporate ladder, regardless of their qualifications or achievements". It is well documented that many societies and organizations exhibit a glass ceiling. In this paper we formally define and study the glass ceiling effect in social networks and propose a natural mathematical model, called the biased preferential attachment model, that partially explains the causes of the glass ceiling effect. This model consists of a network composed of two types of vertices, representing two sub-populations, and accommodates three well known social phenomena: (i) the "rich get richer" mechanism, (ii) a minority-majority partition, and (iii) homophily. We prove that our model exhibits a strong moment glass ceiling effect and that all three conditions are necessary, i.e., removing any one of them will prevent the appearance of a glass ceiling effect. Additionally, we present empirical evidence taken from a mentor-student network of researchers (derived from the DBLP database) that exhibits both a glass ceiling effect and the above three phenomena.

In this paper, we design and analyze information spreading algorithms for dynamic networks with correlated data. In these networks, either the data to be distributed, the data already available at the nodes, or both are correlated. Moreover, nodes' availability and connectivity is dynamic - a scenario typical for wireless networks. Our contribution is twofold. First, although coding schemes for correlated data have been studied extensively, the focus has been on characterizing the rate region in static networks. In an information spreading scheme, however, nodes may communicate by continuously exchanging packets according to some underlying communication model. The main figure of merit is the stopping time - the time required until nodes can successfully decode. While information spreading schemes, such as gossip, are practical, distributed, and scalable, they have only been studied for uncorrelated data. We close this gap by providing techniques to analyze network-coded information spreading in dynamic networks with correlated data. Second, we give a clean framework for oblivious dynamic network models that in particular applies to a multitude of wireless network and communication scenarios. We specify a general setting for the data model and give tight bounds on the stopping times of network-coded protocols in this wide range of settings. En route, we analyze the capacities seen by nodes under a network-coded information spreading protocol, a previously unexplored question. We conclude with extensive simulations, clearly validating the key trends and phenomena predicted in the analysis.

This paper initiates the study of self-adjusting networks (or distributed data structures) whose topologies dynamically adapt to a communication pattern σ. We present a fully decentralized self-adjusting solution called SplayNet. A SplayNet is a distributed generalization of the classic splay tree concept. It ensures short paths (which can be found using local-greedy routing) between communication partners while minimizing topological rearrangements. We derive an upper bound for the amortized communication cost of a SplayNet based on empirical entropies of σ, and show that SplayNets have several interesting convergence properties. For instance, SplayNets features a provable online optimality under special requests scenarios. We also investigate the optimal static network and prove different lower bounds for the average communication cost based on graph cuts and on the empirical entropy of the communication pattern σ. From these lower bounds it follows, e.g., that SplayNets are optimal in scenarios where the requests follow a product distribution as well. Finally, this paper shows that in contrast to the Minimum Linear Arrangement problem which is generally NP-hard, the optimal static tree network can be computed in polynomial time for any guest graph, despite the exponentially large graph family. We complement our formal analysis with a small simulation study on a Facebook graph.

The design of scalable and robust overlay topologies has been a main research subject since the very origins of peerto-peer (p2p) computing. Today, the corresponding optimization tradeoffs are fairly well-understood, at least in the static case and from a worst-case perspective. This paper revisits the peer-to-peer topology design problem from a self-organization perspective. We initiate the study of topologies which are optimized to serve the communication demand, or even self-adjusting as demand changes. The appeal of this new paradigm lies in the opportunity to be able to go beyond the lower bounds and limitations imposed by a static, communication-oblivious, topology. For example, the goal of having short routing paths (in terms of hop count) does no longer conflict with the requirement of having low peer degrees. We propose a simple overlay topology OBST(k) which is composed of k (rooted and directed) Binary Search Trees (BSTs), where k is a parameter. We first prove some fundamental bounds on what can and cannot be achieved optimizing a topology towards a static communication pattern (a static OBST(k)). In particular, we show that the number of BSTs that constitute the overlay can have a large impact on the routing costs, and that a single additional BST may reduce the amortized communication costs from Ω(log n) to O(1), where n is the number of peers. Subsequently, we discuss a natural self-adjusting extension of OBST(k), in which frequently communicating partners are “splayed together”.

The celebrated Perron–Frobenius (PF) theorem is stated for irreducible nonnegative square matrices, and provides a simple characterization of their eigenvectors and eigenvalues. The importance of this theorem stems from the fact that eigenvalue problems on such matrices arise in many fields of science and engineering, including dynamical systems theory, economics, statistics and optimization. However, many real-life scenarios give rise to nonsquare matrices. Despite the extensive development of spectral theories for nonnegative matrices, the applicability of such theories to non-convex optimization problems is not clear. In particular, a natural question is whether the PF Theorem (along with its applications) can be generalized to a nonsquare setting. Our paper provides a generalization of the PF Theorem to nonsquare multiple choice matrices. The extension can be interpreted as representing systems with additional degrees of freedom, where each client entity may choose between multiple servers that can cooperate in serving it (while potentially interfering with other clients). This formulation is motivated by applications to power control in wireless networks, economics and others, all of which extend known examples for the use of the original PF Theorem. We show that the option of cooperation does not improve the situation, in the sense that in the optimum solution, no cooperation is needed, and only one server per client entity needs to work. Hence, the additional power of having several potential servers per client translates into choosing the “best” single server and not into sharing the load between the servers in some way, as one might have expected. The two main contributions of the paper are (i) a generalized PF Theorem that characterizes the optimal solution for a non-convex problem, and (ii) an algorithm for finding the optimal solution in polynomial time. In addition, we extend the definitions of irreducibility and largest eigenvalue of square matrices to nonsquare ones in a novel and non-trivial way, which turns out to be necessary and sufficient for our generalized theorem to hold. To characterize the optimal solution, we use techniques from a wide range of areas. In particular, the analysis exploits combinatorial properties of polytopes, graph-theoretic techniques and analytic tools such as spectral properties of nonnegative matrices and root characterization of integer polynomials.

We design and analyze gossip algorithms for networks with correlated data. In these networks, either the data to be distributed, the data already available at the nodes, or both, are correlated. Although coding schemes for correlated data have been studied extensively, the focus has been on characterizing the rate region in static memory-free networks. In a gossip-based scheme, however, nodes communicate among each other by continuously exchanging packets according to some underlying communication model. The main figure of merit in this setting is the stopping time - the time required until nodes can successfully decode. While Gossip schemes are practical, distributed and scalable, they have only been studied for uncorrelated data. We wish to close this gap by providing techniques to analyze network coded gossip in (dynamic) networks with correlated data. We give a clean framework for oblivious network models that applies to a multitude of network and communication scenarios, specify a general setting for distributed correlated data, and give tight bounds on the stopping times of network coded protocols in this wide range of scenarios.

This paper studies the reception zones of a wireless network in the SINR model with receivers that employ interference cancellation (IC). IC is a recently developed technique that allows a receiver to decode interfering signals, and cancel them from the received signal in order to decode its intended message. We first derive the important topological properties of the reception zones and their relation to high-order Voronoi diagrams and other geometric objects. We then discuss the computational issues that arise when seeking an efficient description of the zones. Our main fundamental result states that although potentially there are exponentially many possible cancellation orderings, and as a result, reception zones, in fact there are much fewer nonempty such zones. We prove a linear bound (hence tight) on the number of zones and provide a polynomial time algorithm to describe the diagram. Moreover, we introduce a novel parameter, the Compactness Parameter, which influences the tightness of our bounds. We then utilize these properties to devise a logarithmic time algorithm to answer point-location queries for networks with IC.

The rules governing the availability and quality of connections in a wireless network are described by physical models such as the signal-to-interference & noise ratio (SINR) model. For a collection of simultaneously transmitting stations in the plane, it is possible to identify a reception zone for each station, consisting of the points where its transmission is received correctly. The resulting SINR diagram partitions the plane into a reception zone per station and the remaining plane where no station can be heard. SINR diagrams appear to be fundamental to understanding the behavior of wireless networks, and may play a key role in the development of suitable algorithms for such networks, analogous perhaps to the role played by Voronoi diagrams in the study of proximity queries and related issues in computational geometry. So far, however, the properties of SINR diagrams have not been studied systematically, and most algorithmic studies in wireless networking rely on simplified graph-based models such as the unit disk graph (UDG) model, which conveniently abstract away interference-related complications, and make it easier to handle algorithmic issues, but consequently fail to capture accurately some important aspects of wireless networks. This article focuses on obtaining some basic understanding of SINR diagrams, their properties and their usability in algorithmic applications. Specifically, we have shown that assuming uniform power transmissions, the reception zones are convex and relatively well-rounded. These results are then used to develop an efficient approximation algorithm for a fundamental point location problem in wireless networks.

In this paper we offer a novel geographical routing algorithm that relies on a well known data structure called Quadtree. Quadtree is an efficient method of mapping a two-dimensional area by recursively partitioning it to disjoint squares. We present a greedy, guaranteed delivery routing algorithm called Greedy-Quadtree-Greedy (GQG). The algorithm is robust to dynamics in the non-Quadtree edges and overcomes local minimums without the use of planarization, face routing, or searching. GQG is a tree-based routing algorithm; it makes greedy forwarding based the location information that is extracted from the Quadtree addresses of the nodes. Bypassing voids is done by a concept of ”tree routing with shortcuts”, which can significantly improve hop stretch and load balancing. As part of the routing system, we present three algorithms: address distribution, network topology discovery, and geographical routing with guaranteed delivery. We keep all broadcasts bounded to one hop, and the nodes' routing state depends on their degree rather than the overall network size. We prove the correctness of the algorithms and present simulations that show the protocol improvement over simple tree-based routing.

Link-state routing protocols are known to be robust and to support shortest paths routing, but they do not support policy-based routing and suffer from scalability problems. Therefore, traditionally, link state protocols are used for intra-domain routing while path (or distance) vector protocols, such as BGP, are used for inter-domain routing where policies and scale are dominant. In this paper we present PSP, a path state protocol for policy-based inter-domain, routing for a network of Autonomous Systems (ASes) such as the Internet. A path state protocol is an extended link-state routing protocol that enables both link-states and path-states advertisements (of path costs). PSP supports policy-based routing similar to BGP and takes into account the Internet-like policies that are based on business relations between ASes. PSP can run on arbitrary multi-region (i.e., hierarchies, areas) networks which reduce global traffic and improve scalability. We provide two versions of the protocol: SLP (Shortest Legal Paths) that guarantees forwarding of messages along the policy-based shortest path and SCLP (Shortest Costumer-preferred Legal Paths) which is easier to implement in a network. We prove the correctness of both algorithms and provide various performance measurements on a real ASes network.

Consider the problem of source coding with side information in large networks with multiple receivers. In this case, standard coding techniques are either prohibitively complex to decode, or require source-network coding separation, resulting in sub-optimal transmission schemes. To alleviate this problem, we offer a joint network-source coding scheme based on matrix sparsification at the code design phase, which allows the terminals to use an efficient decoding procedure (syndrome decoding using LDPC), despite the network coding throughout the network. Via a novel relation between matrix sparsification and rate-distortion theory, we give lower and upper bounds on the best achievable sparsification performance, and analyze our scheme in the limit of weak side information at the receivers. Simulation results motivate the use of this scheme at non-limiting rates as well.

In this paper we study gossip based information spreading with bounded message sizes. We use algebraic gossip to disseminate k distinct messages to all n nodes in a network. For arbitrary networks we provide a new upper bound for uniform algebraic gossip of O((k + log n + D)Δ) rounds with high probability, where D and Δ are the diameter and the maximum degree in the network, respectively. For many topologies and selections of k this bound improves previous results, in particular, for graphs with a constant maximum degree it implies that uniform gossip is order optimal and the stopping time is Θ(k + D).

We study the stopping times of gossip algorithms for network coding. We analyze algebraic gossip (i.e., random linear coding) and consider three gossip algorithms for information spreading Pull, Push, and Exchange. The stopping time of algebraic gossip is known to be linear for the complete graph, but the question of determining a tight upper bound or lower bounds for general graphs is still open. We take a major step in solving this question, and prove that algebraic gossip on any graph of size n is O(Δn) where Δ is the maximum degree of the graph. This leads to a tight bound of Θ(n) for bounded degree graphs and an upper bound of O(n2) for general graphs. We show that the latter bound is tight by providing an example of a graph with a stopping time of Ω(n2). Our proofs use a novel method that relies on Jackson's queuing theorem to analyze the stopping time of network coding; this technique is likely to become useful for future research.

Quorums are a basic construct in solving many fundamental distributed computing problems. One of the known ways of making quorums scalable and efficient is by weakening their intersection guarantee to being probabilistic. This article explores several access strategies for implementing probabilistic quorums in ad hoc networks. In particular, we present the first detailed study of asymmetric probabilistic biquorum systems, that allow to mix different access strategies and different quorums sizes, while guaranteeing the desired intersection probability. We show the advantages of asymmetric probabilistic biquorum systems in ad hoc networks. Such an asymmetric construction is also useful for other types of networks with nonuniform access costs (e.g, peer-to-peer networks). The article includes a formal analysis of these approaches backed up by an extensive simulation-based study. The study explores the impact of various parameters such as network size, network density, mobility speed, and churn. In particular, we show that one of the strategies that uses random walks exhibits the smallest communication overhead, thus being very attractive for ad hoc networks.

Ethernet is widely used in Local Area Networks (LANs) due to its simplicity and cost effectiveness. Today, a great deal of effort is being devoted to extending Ethernet capabilities in order to elevate it from a LAN technology to a ubiquitous networking technology, suitable for deployment in Metropolitan Area Networks (MANs) and even in core, Wide Area Networks (WANs). Current standardized Ethernet networks are based on a spanning tree topology, using the Rapid Spanning Tree Protocol (RSTP) or Multiple Spanning Tree Protocol (MSTP). The spanning tree architecture is useful for avoiding forwarding loops, but may lead to low link utilization and long failure recovery time. In this paper we propose to shift from tree to Directed Acyclic Graph (DAG) topologies and offer a new bridged Ethernet architecture called Orient. Orient is based on assigning an orientation state to each port in the network in order to prevent loops. Thus, the Orient architecture enables a full utilization of all network links and ports, while maintaining simplicity of implementation and compliance with the standardized spanning tree protocols.

The rules governing the availability and quality of connections in a wireless network are described by physical models such as the signal-to-interference & noise ratio (SINR) model. For a collection of simultaneously transmitting stations in the plane, it is possible to identify a reception zone for each station, consisting of the points where its transmission is received correctly. The resulting SINR diagram partitions the plane into a reception zone per station and the remaining plane where no station can be heard.

Quorums are a basic construct in solving many fundamental distributed computing problems. One of the known ways of making quorums scalable and efficient is by weakening their intersection guarantee to being probabilistic. This paper explores several access strategies for implementing probabilistic quorums in ad hoc networks. In particular, we present the first detailed study of asymmetric probabilistic bi-quorum systems and show its advantages in ad hoc networks. The paper includes both a formal analysis of these approaches backed by a simulation based study. In particular, we show that one of the strategies, based on random walks, exhibits the smallest communication overhead.

We pose a new and intriguing question motivated by distributed computing regarding random walks on graphs: How long does it take for several independent random walks, starting from the same vertex, to cover an entire graph? We study the cover time - the expected time required to visit every node in a graph at least once - and we show that for a large collection of interesting graphs, running many random walks in parallel yields a speed-up in the cover time that is linear in the number of parallel walks. We demonstrate that an exponential speed-up is sometimes possible, but that some natural graphs allow only a logarithmic speed-up. A problem related to ours (in which the walks start from some probablistic distribution on vertices) was previously studied in the context of space efficient algorithms for undirected s-t-connectivity and our results yield, in certain cases, an improvement upon some of the earlier bounds.

In recent years random-walk-based algorithms have been proposed for a variety of networking tasks. These proposals include searching, routing, self-stabilization, and query processing in wireless networks, peer-to-peer networks and other distributed systems. This approach is gaining popularity because random walks present locality, simplicity, low-overhead and inherent robustness to structural changes. In this work we propose and investigate an enhanced algorithm that we refer to as random walks with choice. In this algorithm, instead of selecting just one neighbor at each step, the walk moves to the next node after examining a small number of neighbors sampled at random. Our empirical results on random geometric graphs, the model best suited for wireless networks, suggest a significant improvement in important metrics such as the cover time and load-balancing properties of random walks. We also systematically investigate random walks with choice on networks with a square grid topology. For this case, our simulations indicate that there is an unbounded improvement in cover time even with a choice of only two neighbors. We also observe a large reduction in the variance of the cover time, and a significant improvement in visit load balancing.

Many existing systems for sensor networks rely on state information stored in the nodes for proper operation (e.g., pointers to parent in a spanning tree, routing information, etc). In dynamic environments, such systems must adopt failure recovery mechanisms, which significantly increase the complexity and impact the overall performance. In this paper, we investigate alternative schemes for query processing based on random walk techniques. The robustness of this approach under dynamics follows from the simplicity of the process, which only requires the connectivity of the neighborhood to keep moving. In addition we show that visiting a constant fraction of sensor network, say 80%, using a random walk is e#cient in number of messages and su#cient for answering many interesting queries with high quality. Finally, the natural behavior of a random walk, also provide the important properties of load-balancing and scalability.