Internet access through cellular networks is rapidly growing, driven by the great success of the mobile apps paradigm and the overwhelming popularity of social-related multimedia services such as YouTube, Facebook, or even WhatsApp. Understanding the functioning, performance, and traffic generated by these applications is paramount for ISPs, especially for cellular operators, who must manage the huge surge of volume and number of users with the constraints and challenges of cellular networks. In this paper, we study important networking aspects of three popular applications in cellular networks: YouTube, Facebook, and WhatsApp. Our evaluations span the content delivery networks hosting these services, their traffic characteristics, and their performance. The analysis is performed on top of real cellular network traffic monitored at the nationwide cellular network of a major European ISP. Due to privacy issues and given the huge amount of data generated by these applications as well as the large number of monitored customers, the analysis has been done in an online fashion, using a customized big data analytics (BDA) platform called DBStream. We overview DBStream and discuss other potential solutions currently available for traffic monitoring and analysis of big networking data. To the best of our knowledge, this is the first paper providing a complete analysis of popular services in cellular networks, using BDA platforms.

Machine-to-Machine (M2M) network traffic is becoming highly relevant in nowadays cellular networks. The ever-increasing number of M2M devices is heavily modifying the traffic patterns observed in cellular networks, and the interest in discovering and tracking these devices is rapidly growing among operators. In this paper we introduce MTRAC, a complete approach for M2M TRAffic Classification, capable of discovering M2M devices from coarse-grained measurements. MTRAC uses different Machine Learning (ML) algorithms to unveil M2M devices in cellular networks. It relies on very simple traffic descriptors to characterize the communication patterns of each device. These descriptors are robust to traffic encryption techniques, and improve the portability of the MTRAC approach to other network scenarios. MTRAC is implemented on top of DBStream, a novel Data Stream Warehouse which allows to classify M2M devices in an on-line basis, using different temporal and logical traffic aggregations. We study the performance of MTRAC in the on-line classification of more than two months of traffic observed in a operational, nationwide cellular network, comparing different ML algorithms and different traffic aggregation techniques. To the best of our knowledge, MTRAC is the first ML-based approach for automatic M2M device classification in operational cellular networks.

Shared workload optimization is feasible if the set of tasks to be executed is known in advance, as is the case in updating a set of materialized views or executing an extract-transform-load workflow. In this paper, we consider data-intensive workloads with precedence constraints arising from data dependencies. While there has been previous work on identifying common subexpressions and task re-ordering to enable shared scans, in this paper we solve the problem of scheduling shared data-intensive workloads in a cache-oblivious way. Our solution relies on a novel formulation of precedence constrained scheduling with the additional constraint that once a data item is in the cache, all tasks that require this item should execute as soon as possible thereafter. We give an optimal algorithm using A* search over the space of possible orderings, and we propose efficient and effective heuristics that obtain nearly-optimal schedules in much less time. We present experimental results on real-life data warehouse workloads and the TCP-DS benchmark to validate our claims.

The complexity of the Internet has rapidly increased, making it more important and challenging to design scalable network monitoring tools. Network monitoring typically requires rolling data analysis, i.e., continuously and incrementally updating (rolling-over) various reports and statistics over highvolume data streams. In this paper, we describe DBStream, which is an SQL-based system that explicitly supports incremental queries for rolling data analysis. We also present a performance comparison of DBStream with a parallel data processing engine (Spark), showing that, in some scenarios, a single DBStream node can outperform a cluster of ten Spark nodes on rolling network monitoring workloads. Although our performance evaluation is based on network monitoring data, our results can be generalized to other Big Data problems with high volume and velocity.

Network traffic monitoring systems generate high volumes of heterogeneous data streams which have to be processed and analyzed with different time constraints for daily network management operations. Some monitoring applications such as anomaly detection, performance tracking and alerting require fast processing of specific incoming real-time data. Other applications like fault diagnosis and trend analysis need to process historical data and perform deep analysis on generally heterogeneous sources of data. The Data Stream Warehousing (DSW) paradigm provides the means to handle both types of monitoring applications within a single system, providing fast and rich data analysis capabilities as well as data persistence. In this paper, we introduce DBStream, a novel online traffic monitoring system based on the DSW paradigm, which allows fast and flexible analysis across multiple heterogeneous data sources. DBStream provides a novel stream processing language for implementing data processing modules, as well as aggregation, filtering, and storage capabilities for further data analysis. We show multiple traffic monitoring applications running on DBStream, processing real traffic from operational ISPs.

Internet-scale services like YouTube are provisioned by large Content Delivery Networks (CDNs), which push content as close as possible to the end-users to improve their Quality of Experience (QoE) and to pursue their own optimization goals. Adopting space and time variant traffic delivery policies, CDNs serve users' requests from multiple servers/caches at different physical locations and different times. CDNs traffic distribution policies can have a relevant impact on the traffic routed through the Internet Service Provider (ISP), as well as unexpected negative effects on the end-user QoE. In the event of poor QoE due to faulty CDN server selection, a major problem for the ISP is to avoid being blamed by its customers. In this paper we show a real case study in which Google CDN server selection policies negatively impact the QoE of the customers of a major European ISP watching YouTube. We argue that it is extremely important for the ISP to rapidly and automatically detect such events to increase its visibility on the overall operation of the network, as well as to promptly answer possible customer complaints. We therefore present an Anomaly Detection (AD) system for detecting unexpected cache-selection changes in the traffic delivered by CDNs. The proposed algorithm improves over traditional AD approaches by analyzing the complete probability distribution of the monitored features, as well as by self-adapting its functioning to dynamic environments, providing better detection capabilities.

YouTube is the most popular service in today's Internet. Google relies on its massive content delivery network (CDN) to push YouTube videos as close as possible to the end-users, both to improve their watching experience as well as to reduce the load on the core of the network, using dynamic server selection strategies. However, we show that such a dynamic approach can actually have negative effects on the end-user quality of experience (QoE). Through the comprehensive analysis of one month of YouTube flow traces collected at the network of a large European ISP, we report a real case study in which YouTube QoE-relevant degradation affecting a large number of users occurs as a result of Google's server selection strategies. We present an iterative and structured process to detect, characterize, and diagnose QoE-relevant anomalies in CDN distributed services such as YouTube. The overall process uses statistical analysis methodologies to unveil the root causes behind automatically detected problems linked to the dynamics of CDNs' server selection strategies.

In the last years, botnets have become one of the major sources of cyber-crime activities carried out via the public Internet. Typically, they may serve a number of different malicious activities such as Distributed Denial of Service (DDoS) attacks, email spam and phishing attacks. In this paper we validate the Domain Name System (DNS) failure graph approach presented earlier in [1]. In our work we apply this approach in an operational 3G mobile network serving a significantly larger user population.Based on the introduction of stable host identifiers we implement a novel approach to the tracking of botnets over a period of several weeks. Our results reveal the presence of several groups of hosts that are members of botnets. We analyze the host groups exhibiting the most suspicious behavior and elaborate on how these participate in botnets and other malicious activities. In the last part of this work we discuss how the accuracy of our detection approach could be improved in the future by correlating the knowledge obtained from applying our method in different networks.

Data management systems are facing two challenges driven by the requirements of emerging data-intensive applications: more data and less time to process the data. Data volumes continue to increase as new sources and data collecting mechanisms appear. At the same time, these sources tend to be highly dynamic and generate data in the form of a stream, which requires quick reaction to newly arrived data. Traditional data warehouses enable scalable data storage and analytics, including the ability to define nested levels of materialized views. However, views are typically refreshed during downtimes---e.g., every night---which does not meet the latency requirements of many applications. Stream data warehousing is a new data management technology that allows nearly-continuous view refresh as new data arrive, which enables seamless integration of real-time monitoring and business intelligence with long-term data mining. In this paper, we argue that a new benchmark is required for stream warehouses, which should focus on measuring the property that determines the utility of these systems, namely how well they can keep up with the incoming data and guarantee the "freshness" of materialized views.