The rapid expansion of the Internet has enabled cybercriminal operations at unprecedented scale. A recurring tactic is the use of algorithmically generated domains (AGDs) created by domain generation algorithms (DGAs) to orchestrate botnet command-and-control, host phishing content, and distribute malware. Traditional defenses such as blocklists and heuristic rules are brittle against new domains and evolving attacker strategies. We present DeepDGA, a hybrid deep learning architecture that fuses character-level and word-level representations to detect both pseudo-random and dictionary-based DGAs. Character-level embeddings processed by a BiLSTM capture subword patterns and entropy; word-level embeddings derived from a dom2words tokenization and Word2Vec capture linguistic regularities exploited by dictionary-based DGAs. Evaluations on a public benchmark with more than 670,000 domains, including 25 DGA families and benign top-popular domains, demonstrate the superiority of DeepDGA. The model achieves precision and recall above 0.97 for dictionary-based DGAs, and even higher (above 0.98) for pseudo-random DGAs, consistently outperforming state-of-the-art methods across multiple metrics. DeepDGA’s effectiveness, particularly in detecting the more challenging dictionary-based DGAs, highlights the benefit of combining diverse embedding strategies into the same deep learning architecture.

We present TSGFM, a Time Series Graph Foundation Model for zero-shot network monitoring, leveraging spatiotemporal Graph Neural Networks (GNNs) to extract transferable representations across diverse multivariate time series (MTS) domains. Pretrained on heterogeneous time series datasets, TSGFM enables generalization without task-specific fine-tuning, addressing core challenges in dynamic network environments. TSGFM is benchmarked across five real-world MTS datasets and seven zero-shot forecasting scenarios, outperforming five state-of-the-art baselines in six out of seven tasks. Most notably, in zero-shot network monitoring analysis, TSGFM surpasses all competing models by at least 18%, even without any prior exposure to network monitoring data. We further compare TSGFM against leading Time Series Foundation Models (TSFMs), including TimeGPT and TimesFM. TSGFM achieves performance on par with TimeGPT, occasionally surpassing it, and consistently outperforms TimesFM, while using significantly less pretraining data and relying on a much simpler architecture. A detailed analysis of TSGFM’s learned spatial attention patterns reveals domain-specific connectivity structures. In particular, lower attention weights in network monitoring tasks suggest that dense spatial graphs may be unnecessary, opening opportunities for efficient spatial pruning without sacrificing accuracy. This challenges prevailing assumptions favoring fully connected spatiotemporal GNNs. To foster transparency and reproducibility, we release the complete implementation of TSGFM as open source, as well as the tested datasets.

Modern network monitoring applications often rely on traditional machine learning models conceived for specific analysis tasks, which require extensive feature engineering, retraining for different use cases, and struggle with generalization. This lack of adaptability makes the deployment of AI/ML solutions in network monitoring a daunting task, as each new scenario requires significant reconfiguration, manual tuning, and retraining efforts, undermining the broader adoption of AI/ML for network traffic analysis. Time Series Foundation Models (TSFMs), pre-trained on vast and diverse time-series datasets, offer a promising alternative in the network monitoring realm by enabling zero-shot and few-shot adaptability across different monitoring scenarios. In this work, we explore the potential of TSFMs for network monitoring by evaluating their performance in a challenging analysis task: estimating video streaming Quality of Experience (QoE) from encrypted network traffic. Our study assesses the zero-shot and few-shot capabilities of state-of-the-art TSFMs, the impact of time-series granularity, and the role of common traffic features in performance. Using real-world video streaming QoE datasets, we show that TSFMs achieve competitive results in a zero-shot setting – plug-and-play approach, and that their performance can be easily and cost-effectively improved through few-shot learning techniques, even when applied on NetFlow-like features with coarse granularity. Beyond the specific video streaming QoE monitoring application, our findings demonstrate the viability and broader applicability of TSFMs to network monitoring tasks, opening the door to more scalable and generalizable network management solutions.

The increasing complexity and frequency of cyber attacks require Network Intrusion Detection Systems (NIDS) that can adapt to evolving threats. Artificial intelligence (AI), particularly machine learning (ML), has gained increasing popularity in detecting sophisticated attacks. However, their potential lack of interpretability remains a significant barrier to their widespread adoption in practice, especially in security-sensitive areas. In response, various explainable AI (XAI) methods have been proposed to provide insights into the decision-making process. This paper investigates whether these XAI methods, including SHAP, LIME, Tree Interpreter, Saliency, Integrated Gradients, and DeepLIFT, produce similar explanations when applied to ML-NIDS. By analyzing consensus among these methods across different datasets and ML models, we explore whether an agreement exists that could simplify the practical adoption of XAI in cybersecurity, as similar explanations would eliminate the need for rigorous selection processes. Our findings reveal varying degrees of consensus among the methods, suggesting that while some align closely, others diverge significantly, highlighting the need for careful selection and combination of XAI tools to enhance trustworthiness in real-world applications.

Artificial Intelligence (AI), particularly Machine Learning (ML), has become prominent in network monitoring, yet its practical adoption, such as for anomaly and intrusion detection, remains limited. Standard AI/ML methods often exclude experts, reducing trust and hindering practical implementations. Active Learning (AL) allows to integrate admins and their expert knowledge into the ML loop by leveraging expert-labeled data. Together with self-training and automated decisions, AL can enhance model performance, trust, and the ability to adapt to system changes. In this work, we evaluate uncertainty-based AL in network monitoring, offering a comprehensive parameter study for best practices in real-world AI/ML adoption. To this end, we evaluate stream-based and pool-based AL across four datasets for various monitoring use cases and conduct a parameter study on ten uncertainty measures, thereby identifying scenarios benefiting from self-training. By analyzing the impact of admin competence on model performance, we offer actionable guidelines towards the practical implementation of AL.

Early decision-making at the network device is crucial for network security. This entails moving beyond traditional forwarding functions towards more intelligent network devices. One possible strategy to speed up decision-making is to incorporate intelligent traffic analysis functionality directly into the data plane, such that traffic can be analyzed before forwarding. Integrating Artificial Intelligence/Machine Learning (AI/ML) models into the data plane enables quicker processing and reduced reliance on the control plane. We address the development of an AI/ML-driven Intrusion Detection System (IDS) where network devices autonomously make security decisions or defer to an expert oracle, relying on in-band and off-band traffic analysis. Programmable devices, such as those using P4, are essential to enable these functionalities and allow for network device retraining to adapt to changing traffic patterns. We introduce HALIDS, a prototype for in-band AI/ML-IDS using P4, complemented with off-band oracles which support in-network ML-driven classification with more confident classifications, targeting an active learning logic for more accurate in-band analysis. We implement HALIDS using the open source software switch BMv2, and show its operation with real traffic traces publicly available. Evaluation results show that the proposed system is sound and could be implemented in a real network as an efficient and highly adaptive security mechanism.

Network monitoring data generally consists of hundreds of counters periodically collected in the form of time-series, resulting in a complex-to-analyze multivariate time-series (MTS) process. Traditional time-series anomaly detection methods target univariate time-series analysis, which makes the MTS analysis cumbersome and prohibitively complex. We present DC-VAE (Dilated Convolutional -Variational Auto Encoder), a novel approach to anomaly detection in MTS data, leveraging convolutional neural networks (CNNs) and variational autoencoders (VAEs). DC-VAE detects anomalies in MTS data through a single model, exploiting temporal information without sacrificing computational and memory resources. In particular, instead of using recursive neural networks, large causal filters, or many layers, DC-VAE relies on Dilated Convolutions (DC) to capture long and short-term phenomena in the data. We evaluate DC-VAE on the detection of anomalies in the TELCO TELeCOmmunication-networks dataset, a large-scale, multi-dimensional network monitoring dataset collected at an operational mobile Internet Service Provider (ISP), where anomalous events were manually labeled by experts during seven months, at a five-minutes granularity. We benchmark DC-VAE against a broad set of traditional time-series anomaly detectors from the signal processing and machine learning domains. We also evaluate DC-VAE in open, publicly available datasets, comparing its performance against other multivariate anomaly detectors based on deep learning generative models. Results confirm the advantages of DC-VAE, both in terms of MTS data modeling, as well as for anomaly detection. For the sake of reproducibility and as an additional contribution, we make the TELCO dataset publicly available to the community and openly release the code implementing DC-VAE.

Network operators require real-time traffic monitoring insights to provide high performance and security to their customers. It has been shown that artificial intelligence and machine learning (ML) can improve the visibility of telemetry systems, especially with encrypted traffic. However, current solutions cannot cope with high traffic rates and volumes in large-scale networks. To realize the ML-driven network intelligence paradigm at terabit scale, we design Marina, a system that spreads monitoring over a highly efficient data plane, which can extract traffic statistics at line rate, and a powerful ML server, which can run monitoring inference using complex ML models. We apply temporal microaggregation into sub-second time slots and extract moment-based statistics. These allow to flexibly obtain accurate ML-based monitoring decisions during the next time slot. To demonstrate the scalability of our design, we implement and evaluate a Marina data plane prototype on a Barefoot Wedge 100BF-65X P4 switch, which can monitor more than 520,000 concurrent flows at full switching capacity of 6.4 Tbps. We validate the analytics capabilities enabled by our Marina implementation for four ML-driven real-time monitoring tasks with a broad set of standard ML models, achieving comparable or better than state-of-the-art results.

The timely identification of DNS queries to Domain Generation Algorithm (DGA) domains plays a critical role in mitigating malware propagation and its potential impact, especially in thwarting coordinated botnet activity. We introduce Dom2Vec, an innovative approach for swiftly detecting DGA-generated domains by leveraging lexicographic features exclusively derived from the observed domain names in DNS queries. Dom2Vec leverages word embeddings to map tokens extracted from domain names into highly expressive representations. These representations are then combined with a reputation-based scoring system for domain names, which utilizes the co-occurrence frequency of n-grams in relation to a list of whitelisted domains. The fusion of domain embeddings, reputation scores, and other meaningful lexicographic features derived from domain names provides robust domain name representations for AI/ML-driven detection of DGAs. Through experimental evaluation on a dataset comprising 25 distinct families of DGA domains, we demonstrate that Dom2Vec significantly outperforms current state-of-the-art approaches for DGA detection and analysis, improving our previous detection system based on reputation scores by at least 30%, for a false-alarm rate below 1%.

The digital transformation provides industries with unparalleled opportunities for value creation. AI and Machine learning (AI/ML)-driven approaches for data analysis applied to the massive amounts of data steaming from industrial processes can lead to enhanced operation, costs reduction, and powerful decision-making strategies. In this paper we address the problem of Quality Assurance (QA) in industrial manufacturing. We propose Smart Active Sampling (SAS), a new QA sampling strategy for quality inspection outside the production line. Based on the principles of active learning, an AI/ML model trained for quality prediction decides which produced pieces or samples are sent to quality inspection, to further improve its own prediction accuracy. SAS reduces the production of scrap parts due to earlier detection of quality violations. By inspecting a much lower number of samples as compared to traditional random sampling approaches, SAS improves QA efficiency and cuts down quality inspection costs, resulting in an overall smoother operation. We elaborate on some of the challenges faced in smart sampling strategies for quality inspection, describe the main concepts behind SAS, and showcase its application in a real-world manufacturing QA use case, training an AI/ML model for product defect prediction. Compared to a standard random sampling strategy, widely applied today in industrial QA applications, SAS improves model prediction accuracy requiring a significantly lower number of inspected samples, up to five time less samples in the analyzed dataset.

We introduce PHISHWEB, a novel approach to website phishing detection, which detects and categorizes malicious websites through a progressive, multi-layered analysis. PHISHWEB’s detection includes forged domains such as homoglyph and typosquatting, as well as automatically generated domains through DGA technology. The focus of PHISHWEB is on lexicographic-based analysis of the domain name itself, improving applicability and scalability of the approach. Preliminary results on the application of PHISHWEB to multiple open domain-name datasets show precision and recall results above 90%. We additionally extend PHISHWEB’s detection of DGA domains through Machine Learning (ML), using a small set of highly specialized lexicographic domain features. Results on the detection of DGA domains show that, for a false alarm rate below 1%, the ML-extension of PHISHWEB improves non-ML PHISHWEB DGA detector as well as state-of-the-art by at least 60%, realizing precision and recall values of 93.1% and 84.8%, respectively. Finally, we also present preliminary results on the application of PHISHWEB to real, in the wild DNS requests collected at large mobile and fixed-line operational networks, discussing some of the findings.

We propose PHISHWEB, a novel approach to website phishing detection, which detects and categorizes malicious websites through a progressive, multi-layered analysis. PHISHWEB combines and extends different detection approaches proposed in the literature, adding robustness to the identification and visibility into the particular type of deception technique employed by the attacker. We present preliminary results on the application of PHISHWEB to multiple open domain-name datasets, showing precision and recall results above 90% for the specific case of lexicographic-based analysis, improving state-of-the-art detection by more than 60% for Domain Generated Algorithms-driven attacks.

We present DC-VAE, an approach to network anomaly detection in multivariate time-series (MTS), using Variational Auto Encoders (VAEs) and Dilated Convolutional Neural Networks (CNN). DC-VAE detects anomalies in MTS data through a single model, exploiting temporal and spatial MTS information. We showcase DC-VAE in different MTS datasets, and portray its future application in a continual learning framework, exploiting the generative properties of the underlying generative model to deal with continuously evolving data, avoiding catastrophic forgetting. We showcase the functioning of DC-VAE in the event of concept drifts, and propose the application of a novel approach to generative-driven continual learning, introducing the Deep Generative Replay model.

We introduce DeepCrypt, a deep-learning based approach to analyze YouTube adaptive video streaming Quality of Experience (QoE) from the Internet Service Provider (ISP) perspective, relying exclusively on the analysis of encrypted network traffic. Using raw features derived on-line from the encrypted stream of bytes, DeepCrypt infers six different video QoE indicators capturing the user-perceived performance of the service, including the initial playback delay, the number and frequency of rebuffering events, the video playback quality and encoding bitrate, and the number of quality changes. DeepCrypt offers deep visibility into the behavior of the end-user, enabling the fingerprinting and detection of different user actions on the video player, such as video pauses and playback scrubbing (forward, backward, out-of-buffer), offering a complete visibility on the video streaming process from in-network traffic measurements. Evaluations over a large and heterogeneous dataset composed of mobile and fixed-line measurements, using the YouTube HTML5 player, the native YouTube mobile app, as well as a generic HTML5 video player built on top of open source libraries, and considering measurements collected at different ISPs, confirm the out-performance of DeepCrypt over previously used shallow-learning models, and its generalization to different video players and network setups.

Web browsing has become the most important application of the Internet for the end user. When it comes to mobile devices, web services are mainly accessed through apps. This paper tackles the problem of Web Quality of Experience (QoE) in mobile devices, with a specific focus on apps QoE monitoring and analysis, using in-network (encrypted) traffic measurements. Measuring apps QoE is complex, not only from an instrumentation point of view, but also from the heterogeneity of user interactions which might realize substantially different user experience. To this end, we conduct a feasibility study on four specific and popular Android apps and their corresponding web services. Our test automation framework emulates and measures different user interactions commonly executed during an app session, including the app startup, clicking, scrolling, and searching. The resulting traffic is characterized on different dimensions, and machine learning models are trained to identify web services, apps, and user interactions, and to infer their QoE. The proposed models can correctly identify the specific web service and app in 86% of the cases and accurately estimate the associated QoE with small errors. Our preliminary study represents a first step towards an in-network, web QoE monitoring solution for mobile-device apps.

Network-monitoring data commonly arrives in the form of fast and changing data streams. Continuous and dynamic learning is an effective learning strategy when dealing with such data, where concept drifts constantly occur. We propose different stream-based, adaptive learning approaches to analyze network-traffic streams on the fly. We address two major challenges associated to stream-based machine learning and online network monitoring: (i) how to dynamically learn from and adapt to non-stationary data changing over time, and (ii) how to deal with the limited availability of labeled data to continuously tune a supervised-learning model. We introduce ADAM & RAL, two stream-based machine-learning techniques to tackle these challenges. ADAM relies on adaptive memory strategies to dynamically tune stream-based learning models to changes in the input data distribution. RAL combines reinforcement learning with stream-based active-learning to reduce the amount of labeled data needed for continual learning, dynamically deciding on the most informative samples to learn from. We apply ADAM & RAL to the real-time detection of network attacks in Internet network traffic, and show that it is possible to continuously achieve high detection accuracy even under the occurrence of concept drifts, limiting the amount of labeled data needed for learning.

Education Service Providers (ESPs) have a paramount role in the digitization of education, providing reliable devices for students and teachers and high quality Internet access at schools. In this paper, a large-scale, passive, in-device Quality of Experience (QoE) monitoring system is presented, which was deployed into a nationwide network of education-purpose devices. Four months' worth of continuous measurements were conducted by an ESP, covering more than 800 education centers and about 4000 devices, used both in schools and at home. When analyzing the QoE of web sessions in school networks, we identify a fundamental issue with the compatibility of web browsing and video QoE models, which inhibits the successful application of QoE-aware network management for multiple services.

Web browsing is one of the key applications of the Internet, if not the most important one. We address the problem of Web Quality-of-Experience (QoE) monitoring from the ISP perspective, relying on in-network, passive measurements. As a proxy to Web QoE, we focus on the analysis of the well-known SpeedIndex (SI) metric. Given the lack of application-level-data visibility introduced by the wide adoption of end-to-end encryption, we resort to machine-learning models to infer the SI and the QoE level of individual web-page loading sessions, using as input only packet- and flow-level data. In this paper, we study the impact of different end-user device types (e.g., smartphone, desktop, tablet) on the performance of such models. Empirical evaluations on a large, multi-device, heterogeneous corpus of Web-QoE measurements for the most popular websites demonstrate that the proposed solution can infer the SI as well as estimate QoE ranges with high accuracy, using either packet-level or flow-level measurements. In addition, we show that the device type adds a strong bias to the feasibility of these Web-QoE models, putting into question the applicability of previously conceived approaches on single-device measurements. To improve the state of the art, we conceive cross-device generalizable models operating at both packet and flow levels, offering a feasible solution for Web-QoE monitoring in operational, multi-device networks. To the best of our knowledge, this is the first study tackling the analysis of Web QoE from encrypted network traffic in multi-device scenarios.

Large-scale events pose severe challenges to live video streaming service providers, who need to cope with high, peaking viewer numbers and the resulting fluctuating resource demands, keeping high levels of Quality of Experience (QoE) to avoid end-user frustration and churn. In this paper, we analyze a unique dataset consisting of more than a million 2018 FIFA World Cup mobile live streaming sessions, collected at a large national public broadcaster. Different from previous work, we analyze QoE and user engagement as well as their interaction, in dependency to specific soccer match events, which have the potential to trigger flash crowds during a match. Flash crowds are a particular challenge to video service providers, since they cause sudden load peaks and consequently, the likelihood of quality problems. We further exploit the data to model viewer engagement over the course of a soccer match, and show that client counts follow very similar patterns of change across all matches. We believe that the analysis as well as the resulting models are valuable sources of insight for service providers, equipping them with tools for customer-centric resource and capacity management.

The popularity of Artificial Intelligence (AI) – and of Machine Learning (ML) as an approach to AI, has dramatically increased in the last few years, due to its out-standing performance in various domains, notably in image, audio, and natural language processing. In these domains, AI success-stories are boosting the applied field. When it comes to AI/ML for data communication Networks (AI4NETS), and despite the many attempts to turn networks into learning agents, the successful application of AI/ML in networking is limited. There is a strong resistance against AI/ML-based solutions, and a striking gap between the extensive academic research and the actual deployments of such AI/ML-based systems in operational environments. The truth is, there are still many unsolved complex challenges associated to the analysis of networking data through AI/ML, which hinders its acceptability and adoption in the practice. In this positioning paper I elaborate on the most important show-stoppers in AI4NETS, and present a research agenda to tackle some of these challenges, enabling a natural adoption of AI/ML for networking. In particular, I focus the future research in AI4NETS around three major pillars: (i) to make AI/ML immediately applicable in networking problems through the concepts of effective learning, turning it into a useful and reliable way to deal with complex data-driven networking problems; (ii) to boost the adoption of AI/ML at the large scale by learning from the Internet-paradigm itself, conceiving novel distributed and hierarchical learning approaches mimicking the distributed topological principles and operation of the Internet itself; and (iii) to exploit the softwarization and distribution of networks to conceive AI/ML-defined Networks (AIDN), relying on the distributed generation and re-usage of knowledge through novel Knowledge Delivery Networks (KDNs).

Blockchains are typically managed by peer-to-peer (P2P) networks providing the support and substrate to the so-called distributed ledger (DLT), a replicated, shared, and syn-chronized data structure, geographically spread across multiple nodes. The Bitcoin (BTC) blockchain is by far the most well-known DLT, used to record transactions among peers, based on the BTC digital currency. In this paper we focus on the network side of the BTC P2P network, analyzing its nodes from a purely network measurements-based approach. We present a BTC crawler able to discover and track the BTC P2P network through active measurements, and use it to analyze its main properties. Through the combined analysis of multiple snapshots of the BTC network as well as by using other publicly available data sources on the BTC network and DLT, we unveil the BTC P2P network, locate its active nodes, study their performance, and track the evolution of the network over the past two years. Among other relevant findings, we show that (i) the size of the BTC network has remained almost constant during the last 12 months – since the major BTC price drop in early 2018, (ii) most of the BTC P2P network resides in US and EU countries, and (iii) despite this western network locality, most of the mining activity and corresponding revenue is controlled by major mining pools located in China. By additionally analyzing the distribution of BTC coins among independent BTC entities (i.e., single BTC addresses or groups of BTC addresses controlled by the same actor), we also conclude that (iv) BTC is very far from being the decentralized and uncontrolled system it is so much advertised to be, with only 4.5% of all the BTC entities holding about 85% of all circulating BTC coins.

Video streaming is the killer application of the Internet today. In this article, we address the problem of real-time, passive Quality-of-Experience (QoE) monitoring of HTTP Adaptive Video Streaming (HAS), from the Internet-Service-Provider (ISP) perspective - i.e., relying exclusively on in-network traffic measurements. Given the wide adoption of end-to-end encryption, we resort to machine-learning (ML) models to estimate multiple key video-QoE indicators (KQIs) from the analysis of the encrypted traffic. We present ViCrypt, an ML-driven monitoring solution able to infer the most important KQIs for HTTP Adaptive Streaming (HAS), namely stalling, initial delay, video resolution, and average video bitrate. ViCrypt performs estimations in real-time, during the playback of an ongoing video-streaming session, with a fine-grained temporal resolution of just one second. For this, it relies on lightweight, stream-like features continuously extracted from the encrypted stream of packets. Empirical evaluations on a large and heterogeneous corpus of YouTube measurements show that ViCrypt can infer the targeted KQIs with high accuracy, enabling large-scale passive video-QoE monitoring and proactive QoE-aware traffic management. Different from the state of the art, and besides real-time operation, ViCrypt is not bound to coarse-grained KQI-classes, providing better and sharper insights than other solutions. Finally, ViCrypt does not require chunk-detection approaches for feature extraction, significantly reducing the complexity of the monitoring approach, and potentially improving on generalization to different HAS protocols used by other video-streaming services such as Netflix and Amazon.

Network-traffic data commonly arrives in the form of fast data streams; online network-monitoring systems continuously analyze these kinds of streams, sequentially collecting measurements over time. Continuous and dynamic learning is an effective learning strategy when operating in these fast and dynamic environments, where concept drifts constantly occur. In this paper, we propose different approaches for stream-based machine learning, able to analyze network-traffic streams on the fly, using supervised learning techniques. We address two major challenges associated to stream-based machine learning and online network monitoring: (i) how to dynamically learn from and adapt to non-stationary data and patterns changing over time, and (ii) how to deal with the limited availability of ground truth or labeled data to continuously tune a supervised learning model. We introduce ADAM * RAL, two stream-based machine-learning approaches to tackle these challenges. ADAM implements multiple stream-based machine-learning models and relies on an adaptive memory strategy to dynamically adapt the size of the system's learning memory to the most recent data distribution, triggering new learning steps when concept drifts are detected. RAL implements a stream-based active-learning strategy to reduce the amount of labeled data needed for stream-based learning, dynamically deciding on the most informative samples to integrate into the continuous learning scheme. Using a reinforcement learning loop, RAL improves prediction performance by additionally learning from the goodness of its previous sample-selection decisions. We focus on a particularly challenging problem in network monitoring: continuously tuning detection models able to recognize network attacks over time.By continuously learning from and detecting concept drifts within real network measurements, we show that ADAM * RAL can continuously achieve high detection accuracy and limit the amount of training data needed to detect attacks over dynamic network data streams.

After three highly successful editions of the Internet-QoE workshop organized at ACM SIGCOMM 2016, ACM SIGCOMM 2017, and IEEE ICDCS 2018, the goal of the fourth edition of the Internet-QoE workshop is to scale the concepts of Quality of Experience (user satisfaction, user engagement, and behavioral analysis) out of the lab studies context and bring it to the analysis and operation of distributed systems and communication networks, giving a user-centric perspective to the research performed by the MOBICOM community. By fostering an explicit and deep integration of the end-user directly into the design, analysis and management of large-scale operational networks, we expect to reduce the gap between QoE research and its application to future network management paradigms, as well as to provide a more targeted end-user perspective to the research on distributed communication systems. The 4th edition of Internet-QoE also focuses on novel end-user services enabled by next generation technologies such as immersive media (3D, Virtual Reality and Augmented Reality), self-driving cars, intelligent manufacturing systems, Industry 4.0 and tactile Internet, 5G ultra-low-latency mobile networks, and real-time applications.

Over the last years, QUIC (Quick UDP Internet Connections) has become the default protocol for networked communication of Google services, heralded as improved successor of the prevailing Transport Control Protocol (TCP). While the deployment of QUIC is increasing, QUIC is also planned to be the foundation of HTTP/3, the next generation of the HTTP protocols, which drive almost all applications on the Web. Given these developments, this paper aims to raise the awareness of the QoE research community to the increasing presence of QUIC, which likely brings implications for QoE monitoring and management of networked multimedia applications, as well as for the overall QoE research agenda. In particular, a major promise during the introduction of QUIC has been the improvement of the QoE of web-based applications (like browsing and video) by overcoming certain limitations and inefficiencies of TCP. In order to validate this claim, a measurement study was conducted to test whether the promised QoE benefits of QUIC are indeed noticeable for end users of streaming and browsing services. Surprisingly, no evidence for any QoE improvement of QUIC over TCP could be found. This way this paper aims to demonstrate how QoE research can and should successfully address relevant current and future developments on the Internet.

Whether it is for network engineering or business intelligence insight purposes, it is crucial for an Internet Service Provider (ISP) to infer the Quality of Experience (QoE) perceived by the end user during a video streaming session. Specifically, it is important to detect video stalls as soon as they occur, to rapidly take counter-measures such as re-allocating resources more fairly among users. Video stalls fall into two different classes: (i) those caused by poor network conditions and (ii) those caused directly by the user when scrubbing or dragging the video playback forwards or backwards. However, only the former type of stalls degrade the QoE perceived by the end user. Therefore, in this paper we propose a technique to detect and classify stall events by observing the packets associated to a streaming session. We solve a least squares problem to minimize the distance between the estimated chunk's bitrate and the potential bitrate sequence that a plausible playback buffer dynamics would produce. This amounts to finding the maximally likely state sequence for a properly defined Hidden Markov Model. We propose two polynomial dynamic programming algorithms, one of which running in online fashion, computing the exact solution in the ideal case of complete and exact measurement set. We claim that our method is also applicable in an encrypted scenario, since it is robust with respect to the estimation error of a number of parameters, as we show via simulations.

Network Traffic Monitoring and Analysis (NTMA) represents a key component for network management, especially to guarantee the correct operation of large-scale networks such as the Internet. As the complexity of Internet services and the volume of traffic continue to increase, it becomes difficult to design scalable NTMA applications. Applications such as traffic classification and policing require real-time and scalable approaches. Anomaly detection and security mechanisms require to quickly identify and react to unpredictable events while processing millions of heterogeneous events. At last, the system has to collect, store, and process massive sets of historical data for post-mortem analysis. Those are precisely the challenges faced by general big data approaches: Volume, Velocity, Variety, and Veracity. This survey brings together NTMA and big data. We catalog previous work on NTMA that adopt big data approaches to understand to what extent the potential of big data is being explored in NTMA. This survey mainly focuses on approaches and technologies to manage the big NTMA data, additionally briefly discussing big data analytics (e.g., machine learning) for the sake of NTMA. Finally, we provide guidelines for future work, discussing lessons learned, and research directions.

This paper presents a novel adaptation logic for HTTP adaptive streaming (HAS), which achieves not only a high quality of experience (QoE) but also high QoE fairness among independent and heterogeneous clients. The algorithm forces video clients to adapt the requested quality level based on the current network conditions and their individual bit rate requirements, such that the overall quality levels selected by all currently active streaming clients are fairly distributed, i.e., they do not diverge too much. The design of the algorithm is inspired by the well-known transmission control protocol (TCP) congestion control, and drives heterogeneous clients to independently converge on similar quality levels without the need for communicating with each other and/or with a centralized controller in the network. By defining quality levels with equal visual quality, and preparing video representations accordingly, the quality level fairness is extended to QoE fairness. In this paper, the design of the TCP-inspired adaptation logic (TCPAL) is described and a simulative performance evaluation is conducted to compare the QoE and QoE fairness of the proposed algorithm with other HAS adaptation logics. TCPAL is evaluated both in scenarios with stable and fluctuating streaming capacity, and the impact of its parameters is explored. The results suggest that TCPAL performs on par with other HAS adaptation logics in terms of QoE and QoE fairness for low link capacities, but significantly improves the QoE fairness for increased link capacity. Moreover, the fairness achieved by TCPAL does not degrade in situations with fluctuating streaming capacity.

The number of smartphones connected to wireless networks and the volume of wireless network traffic generated by such devices have dramatically increased in the last few years, making it more challenging to tackle wireless network monitoring applications. The high-dimensionality of network data provided by current smartphone devices opens the door to the massive application of machine learning approaches to improve different wireless networking applications. In this paper we study the specific problem of Quality of Experience (QoE) prediction for popular smartphone apps, using machine learning models and in-smartphone measurements. We evaluate and compare different models for the analysis of smartphone generated data, including single models as well as machine learning ensembles such as bagging, boosting and stacking. Results suggest that, while decision-tree based models are the most accurate single models to predict QoE, ensemble learning models, and in particular stacking ones, are capable to significantly increase accuracy prediction and overall classification performance.

P.1203 is a recent standardized model for assessing the Quality of Experience (QoE) of HTTP Adaptive Video Streaming (HAS). However, its complex definition does not allow for a straightforward identification of the underlying assumptions. To overcome this issue, this work investigates the impact of the well-known QoE factors of HAS, namely, initial delay, stalling, and adaptation, on the output QoE score of the model. Therefore, parameter studies are conducted using a reference implementation of P.1203, and the model response to variations of the input QoE factors are compared to results of previous QoE studies in order to get a deeper understanding of the standardized model and its inherent weighting of the QoE factors of HAS.

Internet Service Providers need a thorough understanding of a service to maximize the Quality of Experience (QoE) of their customers by network management. Instead of quantifying the user satisfaction with long and cost-intensive subjective user studies, the QoE can often be estimated with the help of dedicated measurements of application and network parameters. We designed a QoE measurement tool for the popular audio streaming service Spotify that runs inside a Docker software container. The container is able to run headlessly as active measurement probe and emulates a user who is streaming audio files via Spotify. While streaming, network and application parameters are collected that have a high correlation to the user's QoE. The results of the measurements are used to characterize audio streaming in Spotify on application and network layer, and to evaluate important QoE factors.

The application of machine learning models to the analysis of network measurement problems has largely increased in the last decade; however, there is still no clear best-practice or silver bullet approach to address these problems in a general context, and only adhoc and tailored approaches have been evaluated so far. While deep-learning models have provided a major breakthrough in highly-dimensional problems such as image processing, it is difficult to say today which is the best model to address the analysis of large volumes of highly-dimensional data collected in operational networks. In this paper we present a potential solution to fill this gap, exploring the application of ensemble learning models to multiple network measurement problems. We introduce GML Learning, a generic Machine Learning model for the analysis of network measurements. The GML model is a generalization of the well-known stacking approach to ensemble learning, and follows the concepts of the Super Learner model. The Super Learner performs asymptotically as well as the best input base or weak learners, providing a very powerful approach to tackle multiple problems with the same technique. In addition, it defines an approach to minimize over-fitting likelihood during training, using a variant of cross-validation. We deploy the GML model on top of Big-DAMA, a big data analytics framework for network measurement applications. We test the proposed solution in five different and assorted network measurement problems, including detection of network attacks and anomalies, QoE modeling and prediction, and Internet-paths dynamics tracking. Results confirm that the GML model provides better results than any of the single baseline models of the stack, and outperforms traditional bagging and boosting ensemble learning approaches. The GML Learning model opens the door for a generalization of a best-practice technique for the analysis of network measurements.

Monitoring the Quality of Experience (QoE) undergone by cellular network customers has become paramount for cellular ISPs, who need to ensure high quality levels to limit customer churn due to quality dissatisfaction. This paper tackles the problem of QoE monitoring, assessment and prediction in cellular networks, relying on end-user device (i.e., smart-phone) QoS passive traffic measurements and QoE crowdsourced feedback. We conceive different QoE assessment models based on supervised machine learning techniques, which are capable to predict the QoE experienced by the end user of popular smartphone apps (e.g., YouTube and Facebook), using as input the passive in-device measurements. Using a rich QoE dataset derived from field trials in operational cellular networks, we benchmark the performance of multiple machine learning based predictors, and construct a decision-tree based model which is capable to predict the per-user overall experience and service acceptability with a success rate of 91% and 98% respectively To the best of our knowledge, this is the first paper using end-user, in-device passive measurements and machine learning models to predict the QoE of smartphone users in operational cellular networks.

YoMoApp (YouTube Monitoring App) is an Android app to monitor mobile YouTube video streaming on both application- and network-layer. Additionally, it allows to collect subjective Quality of Experience (QoE) feedback of end users. During the development of the app, the stable versions of YoMoApp were already available in the Google Play Store, and the app was downloaded, installed, and used on many devices to monitor streaming sessions. As the app was not advertised in special campaigns or used for dedicated QoE studies, the monitored streaming sessions of this period compose the data set of a large unsupervised field study. The collected data set is evaluated to characterize current mobile YouTube streaming on both application and network layers. Furthermore, the problems and methodology to obtain QoE results from such unsupervised field study are discussed together with the actual QoE results. Correlations between QoE factors are investigated, and the QoE of clusters of similar streaming sessions is analyzed.

The ever-growing population of smartphones connected to mobile networks is changing the cellular traffic ecosystem. The traffic volumes and patterns generated by smartphone apps pose complex challenges to cellular network operators, particularly in terms of detection and diagnosis of network anomalies caused by specific apps. The high-dimensionality of network data provided by current network monitoring systems opens the door to the application of machine learning approaches to improve the detection and classification of network anomalies, but this higher dimensionality comes with an extra data processing overhead. In addition, critical network monitoring applications such as the detection of anomalies require fast mechanisms for on-line analysis of thousands of events per second, as well as efficient techniques for off-line analysis of massive historical data. In this paper we explore the application of big data analytics and big data platforms to the automatic detection of anomalies in mobile networks. We consider ensemble, multi-combined machine learning models to enhance anomaly detection, following a particularly promising model known as Super Learning. Super-learning is an ensemble learning approach which performs asymptotically as well as the best possible weighted combination of multiple learning algorithms, providing a very powerful detection approach. We implement and test different super-learning models on top of Big-DAMA, a big data analytics framework for network monitoring. We test the proposed solution on the detection of traffic anomalies in operational cellular networks, and compare it to other traditional ensemble learning approaches such as bagging and boosting. Results indicate that our approach outperforms traditional ones.

Network security represents a keystone to ISPs, who need to cope with an increasing number of network attacks that put the network's integrity at risk. The high-dimensionality of network data provided by current network monitoring systems opens the door to the massive application of machine learning approaches to improve the detection and classification of network attacks. In this paper we devise a novel attacks detection and classification technique based on semi-supervised Machine Learning (ML) algorithms to automatically detect and diagnose network attacks with minimal training, and compare its performance to that achieved by other well-known supervised learning detectors. The proposed solution is evaluated using real network measurements coming from the WIDE backbone network, using the well-known MAWILab dataset for attacks labeling.

Despite a large body of literature and methods devoted to the analysis of network traffic, the automatic detection and classification of network traffic anomalies still represents a major issue for network operators. The problem becomes even more challenging for cellular ISPs, both due to the ever growing number of connected devices and to the constant deployment of new applications and services prone to performance issues. In this paper we tackle this problem using Machine Learning (ML) approaches: in particular, we devise a system based on Neural Networks to unveil the relations between several monitored traffic features and network anomalies impacting a large number of customers in an operational cellular network. By training a model based on Random Neural Networks (RNN), we provide a fast and accurate anomaly detector and classifier, capable to pinpoint anomalies without assuming any specific traffic model or particular network behavior. The proposed solution is evaluated using synthetically generated data from an operational cellular ISP, drawn from real traffic statistics to resemble the real cellular network traffic. Our RNN model is capable to detect and classify different classes of anomalies with high accuracy and low false alarm rates, even when the volume of such anomalies is small.

A quarter of the world population will be using smartphones to access the Internet in the near future. In this context, understanding the quality of experience (QoE) of popular apps in such devices becomes paramount to cellular network operators, who need to offer high-quality levels to reduce the risks of customers churning for quality dissatisfaction. In this paper, we address the problem of QoE provisioning in smartphones from a double perspective, combining the results obtained from subjective laboratory tests with end-device passive measurements and QoE crowd-sourced feedback obtained in operational cellular networks. The study addresses the impact of both access bandwidth and latency on the QoE of five different services and mobile apps: YouTube, Facebook, Web browsing through Chrome, Google Maps, and WhatsApp. We evaluate the influence of both constant and dynamically changing network access conditions, tackling in particular the case of fluctuating downlink bandwidth, which is typical in cellular networks. As a main contribution, we show that the results obtained in the laboratory are highly applicable in the live scenario, as mappings track the QoE provided by users in real networks. We additionally provide hints and bandwidth thresholds for good QoE levels on such apps, as well as discussion on end-device passive measurements and analysis. The results presented in this paper provide a sound basis to better understand the QoE requirements of popular mobile apps, as well as for monitoring the underlying provisioning network. To the best of our knowledge, this is the first paper providing such a comprehensive analysis of QoE in mobile devices, combining network measurements with users QoE feedback in laboratory tests, and operational networks.

Internet access through cellular networks is rapidly growing, driven by the great success of the mobile apps paradigm and the overwhelming popularity of social-related multimedia services such as YouTube, Facebook, or even WhatsApp. Understanding the functioning, performance, and traffic generated by these applications is paramount for ISPs, especially for cellular operators, who must manage the huge surge of volume and number of users with the constraints and challenges of cellular networks. In this paper, we study important networking aspects of three popular applications in cellular networks: YouTube, Facebook, and WhatsApp. Our evaluations span the content delivery networks hosting these services, their traffic characteristics, and their performance. The analysis is performed on top of real cellular network traffic monitored at the nationwide cellular network of a major European ISP. Due to privacy issues and given the huge amount of data generated by these applications as well as the large number of monitored customers, the analysis has been done in an online fashion, using a customized big data analytics (BDA) platform called DBStream. We overview DBStream and discuss other potential solutions currently available for traffic monitoring and analysis of big networking data. To the best of our knowledge, this is the first paper providing a complete analysis of popular services in cellular networks, using BDA platforms.

A quarter of the world population will be using smartphones to access the Internet in the near future. In this context, understanding the Quality of Experience (QoE) of popular apps in such devices becomes paramount to cellular network operators, who need to offer high quality levels to reduce the risks of customers churning for quality dissatisfaction. In this paper we address the problem of QoE provisioning in smartphones from a double perspective, combining the results obtained from subjective lab tests with end-device passive measurements and QoE crowd-sourced feedback obtained in operational cellular networks. The study addresses the impact of the downlink bandwidth on the QoE of three popular smartphone apps: YouTube, Facebook and Google Maps. As a main contribution, we show that the results obtained in the lab are highly applicable in the live scenario, as mappings track the QoE provided by users in real networks. We additionally provide hints and bandwidth thresholds for good QoE levels on such apps, as well as discussion on end-device passive measurements and analysis. The results presented in this paper provide a sound basis to better understand the QoE requirements of popular mobile apps, as well as for monitoring the underlying provisioning network. To the best of our knowledge, this is the first paper providing such a comprehensive analysis of QoE in mobile devices, combining network measurements with users QoE feedback in lab tests and operational networks.

Machine-to-Machine (M2M) network traffic is becoming highly relevant in nowadays cellular networks. The ever-increasing number of M2M devices is heavily modifying the traffic patterns observed in cellular networks, and the interest in discovering and tracking these devices is rapidly growing among operators. In this paper we introduce MTRAC, a complete approach for M2M TRAffic Classification, capable of discovering M2M devices from coarse-grained measurements. MTRAC uses different Machine Learning (ML) algorithms to unveil M2M devices in cellular networks. It relies on very simple traffic descriptors to characterize the communication patterns of each device. These descriptors are robust to traffic encryption techniques, and improve the portability of the MTRAC approach to other network scenarios. MTRAC is implemented on top of DBStream, a novel Data Stream Warehouse which allows to classify M2M devices in an on-line basis, using different temporal and logical traffic aggregations. We study the performance of MTRAC in the on-line classification of more than two months of traffic observed in a operational, nationwide cellular network, comparing different ML algorithms and different traffic aggregation techniques. To the best of our knowledge, MTRAC is the first ML-based approach for automatic M2M device classification in operational cellular networks.

Shared workload optimization is feasible if the set of tasks to be executed is known in advance, as is the case in updating a set of materialized views or executing an extract-transform-load workflow. In this paper, we consider data-intensive workloads with precedence constraints arising from data dependencies. While there has been previous work on identifying common subexpressions and task re-ordering to enable shared scans, in this paper we solve the problem of scheduling shared data-intensive workloads in a cache-oblivious way. Our solution relies on a novel formulation of precedence constrained scheduling with the additional constraint that once a data item is in the cache, all tasks that require this item should execute as soon as possible thereafter. We give an optimal algorithm using A* search over the space of possible orderings, and we propose efficient and effective heuristics that obtain nearly-optimal schedules in much less time. We present experimental results on real-life data warehouse workloads and the TCP-DS benchmark to validate our claims.

Mobile devices such as smartphones are taking over traditional devices for Internet access in today's scenario, and the near future forecast is overwhelming: by 2016, a quarter of the world population will be using smartphones to access the Internet. In this context, understanding the Quality of Experience (QoE) of popular services in mobile devices becomes paramount for cellular network operators, who need to offer high quality levels to reduce the risks of customers churning for quality dissatisfaction. In this paper we study the problem of QoE provisioning in mobile devices, presenting the results obtained from subjective lab tests performed for popular end-user services accessed through smartphones. Our analysis addresses the impact of access downlink bandwidth on the QoE of four different popular services and mobile apps: Facebook, Web browsing through Chrome, Google Maps, and WhatsApp. The study also considers the characterization of WhatsApp QoE in a real setting, mapping the lab results to large-scale measurements conducted in a major cellular network. The results presented in this paper provide a sound basis for better understanding the QoE requirements of popular services and mobile apps, as well as for dimensioning the underlying provisioning network. To the best of our knowledge, this is the first paper combining QoE lab-test results for mobile devices with large-scale measurements in an operational cellular network.

The DNS protocol has proved to be a valuable means for identifying and dissecting large-scale anomalies in omnipresent Over The Top (OTT) Internet services. In this paper, we present and evaluate a framework for detecting and diagnosing traffic anomalies via DNS traffic analysis. Detection of such anomalies is achieved by monitoring different DNS-related symptomatic features, flagging a warning as soon as one or more of them show a significant change. The investigation of the root causes for such deviations is done by looking at significant changes in a number of diagnostic features (i.e., device manufacturer and OS, requested host name, error codes, etc.), which convey information directly linked to the potential origins of the detected anomalies. For the purpose of detecting significant changes in the time-series of diagnostic features, we propose a scheme based on change point detection applied to the entropy of the considered features. The proposed solution is tested using both real and synthetic data from a nationwide mobile ISP, the latter generated from real traffic statistics to resemble the real mobile network traffic. To show the operational value of the proposed framework, we report the results of the diagnosis in two prototypical cases.

YouTube is the most popular service in the Internet and is increasingly consumed on mobile devices. With emerging adaptive video streaming technology, the question arises whether it should be also employed in the mobile context, which shows different characteristics in terms of display sizes and reliability of Internet connection. This paper compares YouTube QoE on mobile devices for both classical and adaptive video streaming based on a subjective lab experiment, in which different network conditions were emulated. Our results show that adaptive video streaming provides almost excellent results for the poorest network conditions. Thereby, it clearly outperforms classical video streaming, and thus, should be considered to achieve higher QoE in future mobile streaming applications.

The performance of YouTube in cellular networks is crucial to network operators, who try to find a trade-off between cost-efficient handling of the huge traffic amounts and high perceived end-user Quality of Experience (QoE). In this paper we present YoMoApp (YouTube Performance Monitoring Application), an Android application which passively monitors key performance indicators (KPIs) of YouTube adaptive video streaming on end-user smartphones. The monitored KPIs (i.e., player state/events, re-buffering, and video quality levels) can be used to analyze the QoE of mobile YouTube video sessions. YoMoApp is a valuable tool to assess the performance of cellular networks with respect to YouTube traffic, as well as to develop optimizations and QoE models for mobile HTTP adaptive streaming. We try YoMoApp through real subjective QoE lab tests showing that the tool is accurate to capture the experience of end-users watching YouTube on smartphones.

Online Social Networks (OSNs) have rapidly become an integral part of our daily lives, and hundreds of millions of people are nowadays remotely connected trough popular OSNs such as Facebook, Google+, Twitter and WhatsApp. While much has been said and studied about the social aspects of OSNs, little is known about the network side of OSNs, specially regarding their network and traffic footprints, as well as their content delivery infrastructures. In this paper we study these networking aspects of OSNs, vivisecting the most popular OSNs in western countries: Facebook and WhatsApp. By analyzing two large-scale traffic traces collected at the cellular network of a major European ISP, we characterize and compare the networking behavior of Facebook and WhatsApp, considering not only the traffic flows but also the network infrastructures hosting them. Our study serves the main purpose of better understanding how major OSNs are provisioned in today's Internet. To the best of our knowledge, this is the first paper providing such an analysis using large-scale measurements in cellular networks.

In this paper we challenge the applicability of entropy-based approaches for detecting and diagnosis network traffic anomalies, and claim that full statistics (i.e., empirical probability distributions) should be applied to improve the change-detection capabilities. We support our claim by detecting and diagnosing large-scale traffic anomalies in a real cellular network, caused by specific OTT (Over The Top) services and smartphone devices. Our results clearly suggest that anomaly detection and diagnosis based on entropy analysis is prone to errors and misses typical characteristics of traffic anomalies, particularly in the studied scenario.

The complexity of the Internet has rapidly increased, making it more important and challenging to design scalable network monitoring tools. Network monitoring typically requires rolling data analysis, i.e., continuously and incrementally updating (rolling-over) various reports and statistics over highvolume data streams. In this paper, we describe DBStream, which is an SQL-based system that explicitly supports incremental queries for rolling data analysis. We also present a performance comparison of DBStream with a parallel data processing engine (Spark), showing that, in some scenarios, a single DBStream node can outperform a cluster of ten Spark nodes on rolling network monitoring workloads. Although our performance evaluation is based on network monitoring data, our results can be generalized to other Big Data problems with high volume and velocity.

Network traffic monitoring systems generate high volumes of heterogeneous data streams which have to be processed and analyzed with different time constraints for daily network management operations. Some monitoring applications such as anomaly detection, performance tracking and alerting require fast processing of specific incoming real-time data. Other applications like fault diagnosis and trend analysis need to process historical data and perform deep analysis on generally heterogeneous sources of data. The Data Stream Warehousing (DSW) paradigm provides the means to handle both types of monitoring applications within a single system, providing fast and rich data analysis capabilities as well as data persistence. In this paper, we introduce DBStream, a novel online traffic monitoring system based on the DSW paradigm, which allows fast and flexible analysis across multiple heterogeneous data sources. DBStream provides a novel stream processing language for implementing data processing modules, as well as aggregation, filtering, and storage capabilities for further data analysis. We show multiple traffic monitoring applications running on DBStream, processing real traffic from operational ISPs.

Traditional Network Intrusion Detection Systems (NIDSs) rely on either specialized signatures of previously seen attacks, or on expensive and difficult to produce labeled traffic datasets for profiling and training. Both approaches share a common downside: they require the knowledge provided by an external agent, either in terms of signatures or as normal-operation profiles. In this paper we describe UNIDS, an Unsupervised NIDS capable of detecting 0-day attacks, i.e., network attacks for which no signature is yet available, without using any kind of signatures, labeled traffic, or training. UNIDS uses a novel unsupervised outliers detection approach based on Sub-Space Clustering and Multiple Evidence Accumulation techniques to pin-point different kinds of network intrusions and attacks such as DoS/DDoS, probing attacks, propagation of worms, buffer overflows, illegal access to network resources, etc. In this paper we make the strong point that the de-facto approach for NIDS, namely the application of rule-based detection techniques, can be highly harmful for the protected network in case of 0-day attacks. In contrast, we show how UNIDS can work as a complementary system to current NIDS to detect the occurrence of previously unseen attacks. For doing so, we compare the performance of a standard rule-based NIDS against UNIDS to detect 0-day attacks in the well-known KDD99 dataset. In addition, we also compare the performance of UNIDS against other popular unsupervised detection techniques to detect attacks in traces collected at two operation networks.

Internet-scale services like YouTube are provisioned by large Content Delivery Networks (CDNs), which push content as close as possible to the end-users to improve their Quality of Experience (QoE) and to pursue their own optimization goals. Adopting space and time variant traffic delivery policies, CDNs serve users' requests from multiple servers/caches at different physical locations and different times. CDNs traffic distribution policies can have a relevant impact on the traffic routed through the Internet Service Provider (ISP), as well as unexpected negative effects on the end-user QoE. In the event of poor QoE due to faulty CDN server selection, a major problem for the ISP is to avoid being blamed by its customers. In this paper we show a real case study in which Google CDN server selection policies negatively impact the QoE of the customers of a major European ISP watching YouTube. We argue that it is extremely important for the ISP to rapidly and automatically detect such events to increase its visibility on the overall operation of the network, as well as to promptly answer possible customer complaints. We therefore present an Anomaly Detection (AD) system for detecting unexpected cache-selection changes in the traffic delivered by CDNs. The proposed algorithm improves over traditional AD approaches by analyzing the complete probability distribution of the monitored features, as well as by self-adapting its functioning to dynamic environments, providing better detection capabilities.

Today's Internet consists of massive scale web services and Content Delivery Networks (CDNs). This paper sheds light on the way major Internet-scale web services content is hosted and delivered. By analyzing a full month of HTTP traffic traces collected at the mobile network of a major European ISP, we characterize the paradigmatic case of Facebook, considering not only the traffic flows but also the main organizations and CDNs providing them. Our study serves the main purpose of better understanding how major web services are provisioned in today's Internet, paying special attention to the temporal dynamics of the service delivery and the interplays between the involved hosting organizations. To the best of our knowledge, this is the first paper providing such an analysis in mobile networks.

WhatsApp, the new giant in instant multimedia messaging in mobile networks is rapidly increasing its popularity, taking over the traditional SMS/MMS messaging. In this paper we present the first large-scale characterization of WhatsApp, useful among others to ISPs willing to understand the impacts of this and similar applications on their networks. Through the combined analysis of passive measurements at the core of a national mobile network, worldwide geo-distributed active measurements, and traffic analysis at end devices, we show that: (i) the WhatsApp hosting architecture is highly centralized and exclusively located in the US; (ii) video sharing covers almost 40% of the total WhatsApp traffic volume; (iii) flow characteristics depend on the OS of the end device; (iv) despite the big latencies to US servers, download throughputs are as high as 1.5 Mbps; (v) users react immediately and negatively to service outages through social networks feedbacks.

YouTube is the most popular service in today's Internet. Google relies on its massive content delivery network (CDN) to push YouTube videos as close as possible to the end-users, both to improve their watching experience as well as to reduce the load on the core of the network, using dynamic server selection strategies. However, we show that such a dynamic approach can actually have negative effects on the end-user quality of experience (QoE). Through the comprehensive analysis of one month of YouTube flow traces collected at the network of a large European ISP, we report a real case study in which YouTube QoE-relevant degradation affecting a large number of users occurs as a result of Google's server selection strategies. We present an iterative and structured process to detect, characterize, and diagnose QoE-relevant anomalies in CDN distributed services such as YouTube. The overall process uses statistical analysis methodologies to unveil the root causes behind automatically detected problems linked to the dynamics of CDNs' server selection strategies.

The popularity of web-based services and multimedia applications like YouTube, Google Web Search, Facebook, and a bewildering range of Internet applications has taken HTTP back to the pole position on end-user traffic consumption. Today's Internet users exchange most of their content via HTTP. In this paper we address the problem of on-line HTTP traffic classification from network measurements. Building on the results provided by HTTPTag, a flexible system for on-line HTTP classification, we present and explore Mini-IPC. Mini-IPC is a minimalist approach for classifying HTTP flows using only the IP addresses of the servers hosting the corresponding content. Using one full week of HTTP traffic traces collected at the mobile broadband network of a major European ISP, we investigate to which extent the most popular HTTP-based services are hosted by well-defined sets of IP addresses, and evaluate the performance of Mini-IPC to classify these services using IPs only.

YouTube, the killer application of today's Internet, is changing the way ISPs and network operators manage quality monitoring and provisioning on their IP networks. YouTube is currently the most consumed Internet application, accounting for more than 30% of the overall Internet's traffic worldwide. Coupling such an overwhelming traffic volume with the ever intensifying competition among ISPs is pushing operators to integrate Quality of Experience (QoE) paradigms into their traffic management systems. The need for automatic QoE assessment solutions becomes even more critical in mobile broadband networks, where over-provisioning solutions can not be foreseen and bad user experience translates into churning clients. This paper presents a complete study on the problem of YouTube Quality of Experience monitoring and assessment in mobile networks. The paper considers not only the QoE analysis, modeling and assessment based on real users' experience, but also the passive monitoring of the quality provided by the ISP to its end-customers in a large mobile broadband network.

Driven by the well-known limitations of port and payload-based analysis techniques, the use of Machine Learning for Internet traffic analysis and classification has become a fertile research area during the past half-decade. In this paper we introduce a novel unsupervised approach to identify different classes of IP flows sharing similar characteristics. The unsupervised analysis is accomplished by means of robust clustering techniques, using Sub-Space Clustering, Evidence Accumulation, and Hierarchical Clustering algorithms to explore inter-flows structure. Our approach permits to identify natural groupings of traffic flows, combining the evidence of data structure provided by different partitions of the same set of traffic flows. The technique is further used to build an automatic flow classification model, using a semi-supervised-learning-based approach. The approach uses only a reduced fraction of labeled flows to map the identified clusters into their associated most-probable originating application, which strongly simplifies its calibration. We evaluate the performance of our techniques using real traffic traces, additionally comparing their performance against previously proposed clustering-based classification methods.

International audience

Traffic Engineering (TE) has become a challenging mechanism for network management and resources optimization due to the uncertainty and the difficulty to predict current traffic patterns. Recent works have proposed robust optimization techniques to cope with uncertain traffic, computing a stable routing configuration that is immune to demand variations within certain uncertainty set. However, using a single routing configuration for long-time periods can be highly inefficient. Even more, the presence of abnormal and malicious traffic has magnified the network operation problem, claiming for solutions which not only deal with traffic uncertainty but also allow to identify faulty traffic. In this paper, we propose two complementary methods to tackle both problems. Based on expected traffic patterns, we adapt the uncertainty set and build a multi-hour yet robust routing scheme that outperforms the stable approach. For the case of anomalous and unexpected traffic, we propose a fast anomaly detection/isolation algorithm which relies on a novel linear spline-based model of traffic demands to identify traffic problems and decide routing changes. This algorithm is optimal in the sense that it minimizes the decision delay for a given mean false alarm rate and false isolation probabilities. Both proposals are validated using real traffic data from two Internet backbone networks.