As telecommunication networks have become large complex systems, it is recognized today that there is an urgent need for providing comprehensive views about how the network operates, instead of fragmented reports about the different elements in the network. We propose a system that holistically analyzes the network along carefully balanced dimensions, both by using historic data and by using data across multiple LTE RAN networks. Our system learns baseline behaviours and provides insights to the operators alerting them about deviations from this learnt behaviour. The system can also help operators by discovering network configurations that lead to good radio performance, which can be used to bootstrap the network configuration process. All the learnt behaviour is made available globally through a global sharing service. We validated our approach using real 4G RAN network data, and show that we discover behaviours in the network which would not be obvious to the operator otherwise.

Much work has been done in the area of telecommunications transmission parameters and, in particular, Channel Quality Indicators, to analyse the best ways to improve the performance and quality of communications. However, there is a gap in published papers when it comes to real-world experiences with CQI reports and network-side analysis of transmission parameters that can provide information about user equipment and network elements behaviour. In this paper, we present our experiences with a real-world analysis of dense LTE networks, focusing on two main aspects - what information can CQI reports give us about the behaviour of user equipment in the network, and what information do they provide in relation to the individual network elements. We show what insights can be derived from a network wide statistical analysis of CQI reports and how these insights can be further used by operators to understand potential problems in their network.

In many domains such as Telecom various scenarios necessitate the processing of large amounts of data using statistical and machine learning algorithms. A noticeable effort has been made to move the data management systems into MapReduce parallel processing environments such as Hadoop and Pig. Nevertheless these systems lack the features of advanced machine learning and statistical analysis. Frame-works such as Mahout on top of Hadoop support machine learning but their implementations are at the preliminary stage. For example Mahout does not provide Support Vector Machine (SVM) algorithms and it is difficult to use. On the other hand traditional statistical software tools such as R containing comprehensive statistical algorithms for advanced analysis are widely used. But such software can only run on a single computer and therefore it is not scalable. In this paper we propose an integrated solution RPig which takes the advantages of R (for machine learning and statistical analysis capabilities) and parallel data processing capabilities of Pig. The RPig framework offers a scalable advanced data analysis solution for machine learning and statistical analysis. Analysis jobs can be easily developed with RPig script in high level languages. We describe the design implementation and an eclipse-based RPigEditor for the RPig framework. Using application scenarios from the Telecom domain we show the usage of RPig and how the framework can significantly reduce the development effort. The results demonstrate the scalability of our framework and the simplicity of deployment for analysis jobs.

With the explosion of broadband penetration and residential gateways, users' interest in offering user generated content and services from their own home has become more and more pronounced. Home Area Networks (HANs) with rich user content are therefore emerging at the “edge” of the telecom and Internet service providers' networks. Home consumer devices such as Home Gateways (HGs) and femto base stations act as gateways for end user generated content and services. In this context, our paper addresses the important issue of how to address these devices and the users' content and services with ease, without introducing yet another addressing scheme, but instead building on top of widely used telecom network addressing. Our new addressing paradigm is based on a lightweight DHT which augments classic telecommunication addressing schemas such as SS7 and SIP. The experimental results for our prototype are based on simulations conducted on a ModelNet cluster and the results have shown that our implementation scales well while exhibiting good fault tolerant properties. Our implementation has been released to the open source community on sourceforge.net.

Quality of Service (QoS) monitoring of end-user services is an integral and indispensable part of service management. However in large, heterogeneous and complex networks where there are many services, many types of end-user devices, and huge numbers of subscribers, it is not trivial to monitor QoS and estimate the status of Service Level Agreements (SLAs). Furthermore, the overwhelming majority of end-terminals do not provide precise information about QoS which aggravates the difficulty of keeping track of SLAs. In this paper, we describe a solution that combines a number of techniques in a novel and unique way to overcome the complexity and difficulty of QoS monitoring. Our solution uses a model driven approach to service modeling, data mining techniques on small sample sets of terminal QoS reports (from “smarter” end-user devices), and network level key performance indicators (N-KPIs) from probes to address this problem. Service modeling techniques empowered with a modeling engine and a purpose-built language hide the complexity of SLA status monitoring. The data mining technique uses its own engine and learnt data models to estimate QoS values based on N-KPIs, and feeds the estimated values to the modeling engine to calculate SLAs. We describe our solution, the prototype and experimental results in the paper.

In this paper we present an empirical study of a workload gathered by crawling the eDonkey network --- a dominant peer-to-peer file sharing system --- for over 50 days.We first confirm the presence of some known features, in particular the prevalence of free-riding and the Zipf-like distribution of file popularity. We also analyze the evolution of document popularity.We then provide an in-depth analysis of several clustering properties of such workloads. We measure the geographical clustering of peers offering a given file. We find that most files are offered mostly by peers of a single country, although popular files don't have such a clear home country.We then analyze the overlap between contents offered by different peers. We find that peer contents are highly clustered according to several metrics of interest.We propose to leverage this property by allowing peers to search for content without server support, by querying suitably identified semantic neighbours. We find via trace-driven simulations that this approach is generally effective, and is even more effective for rare files. If we further allow peers to query both their semantic neighbours, and in turn their neighbours' neighbours, we attain hit rates as high as over 55% for neighbour lists of size 20.

This paper presents GosSkip, a self organizing and fully distributed overlay that provides a scalable support to data storage and retrieval in dynamic environments. The structure of GosSkip, while initially possibly chaotic, eventually matches a perfect set of Skip-list-like structures, where no hash is used on data attributes, thus preserving semantic locality and permitting range queries. The use of epidemic-based protocols is the key to scalability, fairness and good behavior of the protocol under churn, while preserving the simplicity of the approach and maintaining O(log(N)) state per peer and O(log(N)) routing costs. In addition, we propose a simple and efficient mechanism to exploit the presence of multiple data items on a single physical node. GosSkip's behavior in both a static and a dynamic scenario is further conveyed by experiments with an actual implementation and real traces of a peer to peer workload

This paper presents a novel adaptation mechanism that allows every node of a gossip-based broadcast algorithm to adjust the rate of message emission 1) to the amount of resources available to the nodes within the same broadcast group and 2) to the global level of congestion in the system. The adaptation mechanism can be applied to all gossip-based broadcast algorithms we know of and makes their use more realistic in practical situations where nodes have limited resources whose quantity changes dynamically with time without decreasing the reliability. 1

Gossip-based broadcast algorithms, a family of probabilistic broadcast algorithms, trade reliability guarantees against "scalability" properties. Scalability in this context has usually been expressed in terms of message throughput and delivery latency, but there has been little work on how to reduce the memory consumption for membership management and message buffering at large scale.This paper presents lightweight probabilistic broadcast ( lpbcast ), a novel gossip-based broadcast algorithm, which complements the inherent throughput scalability of traditional probabilistic broadcast algorithms with a scalable memory management technique. Our algorithm is completely decentralized and based only on local information: in particular, every process only knows a fixed subset of processes in the system and only buffers fixed "most suitable" subsets of messages. We analyze our broadcast algorithm stochastically and compare the analytical results both with simulations and concrete implementation measurements.

The growing interest in peer-to-peer applications has underlined the importance of scalability in modern distributed systems. Not surprisingly, much research effort has been invested in gossip-based broadcast protocols. These trade the traditional strong reliability guarantees against very good "scalability" properties. Scalability is in that context usually expressed in terms of throughput and delivery latency, but there is only little work on how to reduce the overhead of membership management on a large scale. The paper presents Lightweight Probabilistic Broadcast (lpbcast), a novel gossip-based broadcast algorithm which preserves the inherent throughput scalability of traditional gossip-based algorithms and adds a notion of membership management scalability: every process only knows a random subset of fixed size of the processes in the system. We formally analyze our broadcast algorithm in terms of scalability with respect to the size of individual views, and compare the analytical results both with simulations and concrete measurements.

We present in this paper a general garbage collection scheme that reduces the "noise" in gossip-based broadcast algorithms. In short, our garbage collection scheme uses a simple heuristic to trade "useless" messages with "useful" ones. Used with a given gossip-based broadcast algorithm, a given size of buffers, and a given number of disseminated messages (e.g., per gossip round), our garbage collection scheme provides higher overall reliability than more conventional schemes. We illustrate our approach through two algorithms: bimodal multicast (pbcast) and lightweight probabilistic broadcast (lpbcast). Our scheme is based on the intuitive idea of discarding messages according to their "age". The "age" of a message represents the number of times the message has been retransmitted.