Satellite Communication (SatCom) offers internet connectivity where traditional infrastructures are too expensive to deploy. When using satellites in a geostationary orbit, the distance from Earth forces a round-trip time of at least 550 ms. Coupled with the constrained capacity of the physical link, this challenges the traditional internet access quality we are used to. In this paper, we present a complete passive characterization of the traffic carried by an operational SatCom provider. With this unique vantage point, we observe the performance of the SatCom technology, as well as the usage habits of subscribers in different countries in Europe and Africa. We highlight the implications of such technology on Internet usage and functioning, and we pinpoint technical challenges due to the CDN and DNS resolution issues, while discussing possible optimizations that the ISP could implement to improve the service offered to SatCom subscribers. We complete the characterization of the adoption and performance of newer protocols with a focus on IPv6 and QUIC.

• Neural Network feature selection methodology for both numerical and categorical data. • Embedded feature selection provides model and feature subset in a single training. • Novel feature embedding for dealing with high-cardinality categorical features. • Validation of the methodology on several open and real industry dataset. • Comparison with several state-of-the-art feature selection methodologie. In an era of effortless data collection, the impact of machine learning — especially neural networks (NNs) — is undeniable. As datasets grow in size and complexity, efficiently handling mixed data types, including categorical and numerical features, becomes critical. Feature encoding and selection play a key role in improving NN performance, efficiency, interpretability, and generalisation. This paper presents GLEm-Net (Grouped Lasso with Embeddings Network), a novel NN-based approach that seamlessly integrates feature encoding and selection directly into the training process. GLEm-Net uses embedding layers to process categorical features with high cardinality, simplifying the model and improving generalisation. By extending the grouped Lasso regularisation to explicitly consider categorical features, GLEm-Net automatically identifies the most relevant features during training and returns them to the analyst. We evaluate GLEm-Net on open and proprietary industry datasets and compare it to state-of-the-art feature selection methodologies. Results show that GLEm-Net adapts to each dataset by allowing the NN to directly select subsets of most important features, offering on par performance with the best state-of-the-art feature selection methods, while eliminating the need for the external feature encoding and selection steps that are now incorporated in the NN training stage.

network telescope is a range of IP addresses that host no services. Millions of bots and scanners contact it to look for vulnerable systems, and the traffic it exposes is fundamental to understanding malicious activities. The visibility a telescope offers depends on its size and geolocation, and merging the information from multiple telescopes could help increase visibility and uncover more malicious activities. However, sharing raw telescope data is complicated, calling for solutions that allow one to directly share the knowledge rather than the data obtained from multiple deployments. In this paper, we explore the application of Federated Learning (FL) to create and share such global knowledge from the malicious activities seen in distributed telescopes. For that, we introduce FedScope, an FL-based solution for generatinghost embeddingsin a distributed way. We compare FedScope to local and distributed alternatives in downstream tasks, such as sender classification or coordinated activities detection. We show that FedScope (i) produces embeddings of equal or higher quality than those of a single telescope; (ii) increases coverage, allowing the global model to monitor more malicious actors; (iii) avoids the sharing of the raw data, limiting exchanged data.

This study delves into the mechanisms that spark user curiosity driving active engagement within public Telegram groups. By analyzing approximately 6 million messages from 29,196 users across 409 groups, we identify and quantify the key factors that stimulate users to actively participate (i.e., send messages) in group discussions. These factors include social influence, novelty, complexity, uncertainty, and conflict, all measured through metrics derived from message sequences and user participation over time. After clustering the messages, we apply explainability techniques to assign meaningful labels to the clusters. This approach uncovers macro categories representing distinct curiosity stimulation profiles, each characterized by a unique combination of various stimuli. Social influence from peers and influencers drives engagement for some users, while for others, rare media types or a diverse range of senders and media sparks curiosity. Analyzing patterns, we found that user curiosity stimuli are mostly stable, but, as the time between the initial message increases, curiosity occasionally shifts. A graph-based analysis of influence networks reveals that users motivated by direct social influence tend to occupy more peripheral positions, while those who are not stimulated by any specific factors are often more central, potentially acting as initiators and conversation catalysts. These findings contribute to understanding information dissemination and spread processes on social media networks, potentially contributing to more effective communication strategies.

Network administrators have long relied on passive measurement to detect malicious activity, diagnose misconfigurations, and ensure network health. Under the assumption that threats and issues originate externally, prior studies [1] have predominantly focused on the analysis of inbound traffic — i.e., traffic initiated by external hosts and targeting internal destinations. Conversely, outbound traffic — i.e., originating from internal hosts toward external destinations — has received comparatively less attention, despite carrying strong indicators of security-relevant anomalies [2].

This paper presents a real-world deployment case study on using unsupervised anomaly detection for Anti-Money Laundering (AML).Using more than 2 billion anonymized bank transactions that Intesa Sanpaolo, a primary Italian financial institution, registered over 8 months, we developed, tuned and deployed a machine learning pipeline in production.Experts from Intesa Sanpaolo validated the performance of our approach against the institution's traditional rule-based system and checked new real-world cases the system allowed them to identify.Besides increasing both precision and recall by a factor of 6 in the detection of high-risk cases, our pipeline raises 200+ additional alerts during the 8-month period, manually identified by branch managers, but missed by the rulebased system.More importantly, a manual inspection of 100 new unseen cases revealed 28 significant previously unreported cases.The pipeline, now fully deployed in Intesa Sanpaolo's Transaction Monitoring system, highlights the advantages of machine learning over traditional approaches typically adopted in this traditionally very conservative sector.

Recently we have witnessed the explosion of proposals that, inspired by Language Models like BERT, exploit Representation Learning models to create traffic representations. All of them promise astonishing performance in encrypted traffic classification (up to 98% accuracy). In this paper, with a networking expert mindset, we critically reassess their performance. Through extensive analysis, we demonstrate that the reported successes are heavily influenced by data preparation problems, which allow these models to find easy shortcuts - spurious correlation between features and labels - during fine-tuning that unrealistically boost their performance. When such shortcuts are not present - as in real scenarios - these models perform poorly. We also introduce Pcap-Encoder, an LM-based representation learning model that we specifically design to extract features from protocol headers. Pcap-Encoder appears to be the only model that provides an instrumental representation for traffic classification. Yet, its complexity questions its applicability in practical settings. Our findings reveal flaws in dataset preparation and model training, calling for a better and more conscious test design. We propose a correct evaluation methodology and stress the need for rigorous benchmarking.

Web services, as the most ubiquitous form of online services, have consistently attracted research attention due to privacy concerns. Although VPNs and anonymous communication methods can partially protect users’ online privacy, advancements in website fingerprinting (WF) attacks still exploit the spatio-temporal characteristics of Web resource transmission to identify Web services. The challenge lies in defending against WF attacks efficiently, with limited bandwidth costs. Server-side WF defenses, deployed on Web servers, can achieve end-to-end obfuscation across both clients and servers. However, existing defenses often consume significant bandwidth and require additional removal operations on the client side. Given the growing use of QUIC with HTTP/3 and the need for robust privacy protections, this paper introduces an asymmetric server-side WF defense scheme using State-Transition Adversarial Perturbations (STAP). STAP introduces the concept of latent resource-state transitions, which represent hidden patterns in resource transmission. Utilizing perturbation models containing these transitions, STAP subtly alters traffic through packet padding and insertion, with inherent transport layer encryption enhancing the concealment. STAP can operate independently, removing the necessity for user involvement. Experimental results demonstrate that STAP outperforms other schemes, achieving reductions in True Positive Rate (TPR) by up to 22% and reductions in bandwidth overhead by up to 30%.

In response to growing concerns about user privacy, legislators have introduced new regulations and laws, such as the General Data Protection Regulation (GDPR) in the European Union and the California Consumer Privacy Act (CCPA), which force websites to obtain user consent before activating any personal data collection. The cornerstone of this consent-seeking process involves the use of Privacy Banners, the technical tools to collect users’ approval for data collection practices. Consent management platforms (CMPs) have emerged as practical solutions to simplify the configuration and management of such privacy banners for website administrators, allowing them to outsource the complexities of managing user consent and activating advertising features. This article presents a detailed and longitudinal analysis of the evolution of CMPs spanning 9 years. We take a twofold perspective: firstly, thanks to the HTTP Archive dataset, we provide insights into the growth, market share, and geographical spread of CMPs. Noteworthy observations include the substantial impact of the GDPR on the proliferation of CMPs in Europe, where more than 40% of websites currently adopt a CMP. Secondly, we analyse millions of user interactions with a medium-sized CMP present in thousands of websites worldwide. We observe how even small changes in the design of Privacy Banners have a critical impact on the user’s giving or denying one’s consent to data collection. For instance, over 60% of users do not consent when offered a simple “one-click reject-all” option. Conversely, when opting out requires more than one click, about 90% of users prefer to simply give their consent. This hints that their main objective is to eliminate the annoying privacy banner rather than make an informed decision. Curiously, we observe that iOS users exhibit a higher tendency to accept cookies compared with Android users, possibly indicating greater confidence in the privacy offered by Apple devices. We believe that the findings of this article contribute to a deeper understanding of the multifaceted interactions between privacy regulations, technological solutions and user choices in the evolving Web ecosystem. We also show that the availability of large open datasets, although not explicitly designed and collected for our goals, is fundamental to exploring different angles of the internet evolution over time. For this, we make the data and code used in this work available to the community. 1

Web data are often modelled as complex networks in which entities interact and form communities. Nevertheless, web data evolves over time, and network communities change alongside it. This makes Community Detection (CD) in dynamic graphs a relevant problem, calling for evolutionary CD algorithms. The choice and evaluation of such algorithm performance is challenging because of the lack of a comprehensive set of benchmarks and specific metrics. To address these challenges, we propose CoDÆN—Community Detection Algorithms in Evolving Networks—a benchmarking framework for evolutionary CD algorithms in dynamic networks, that we offer as open source to the community. CoDÆN allows us to generate synthetic community-structured graphs with known ground truth and design evolving scenarios combining nine basic graph transformations that modify edges, nodes, and communities. We propose three complementary metrics (i.e., Correctness, Delay, and Stability) to compare evolutionary CD algorithms. Armed with CoDÆN, we consider three evolutionary modularity-based CD approaches, dissecting their performance to gauge the trade-off between the stability of the communities and their correctness. Next, we compare the algorithms in real Web-oriented datasets, confirming such a trade-off. Our findings reveal that algorithms that introduce memory in the graph maximise stability but add delay when abrupt changes occur. Conversely, algorithms that introduce memory by initialising the CD algorithms with the previous solution fail to identify the split and birth of new communities. These observations underscore the value of CoDÆN in facilitating the study and comparison of alternative evolutionary community detection algorithms.

Satellite Communication (SatCom) is the only choice to access the Internet in remote regions and is characterized by extreme latency and constrained capacity. For SatCom operators, it is thus fundamental to monitor the Quality of Experience (QoE) of subscribers, to measure their satisfaction, spot anomalies and optimize the peculiar network setup. The Web has become the primary source of Internet content, and Web browsing is the main activity of internauts. This paper addresses the challenge of monitoring Web QoE in SatCom environments, proposing a tailored system that employs a supervised approach to predict Web QoE using passive measurements. The system collects training data through Test Agents that mimic real subscribers' traffic patterns and uses them to build Machine Learning (ML) models that predict performance metrics. The findings demonstrate the feasibility of monitoring Web QoE in SatCom environments, with limitations on website applicability and temporal stability. The need for periodic data generation and the development of a general machine learning model for unseen websites remain open challenges. This research contributes to enhancing web browsing experiences in SatCom and expanding understanding of Web QoE monitoring in diverse network settings.

Security logs are the key to understanding attacks and diagnosing vulnerabilities. Often coming in the form of text logs, their analysis remains a daunting challenge. Language Models (LMs) have demonstrated unmatched potential in understanding natural and programming languages. The question arises as to whether and how LMs could be also used to automatise the analysis of security logs. We here systematically study how to benefit from the state-of-the-art LM to support the analysis of text-like Unix shell attack logs automatically. For this, we thoroughly designed LogPrécis. LogPrécis receives as input malicious shell sessions. It then automatically identifies and assigns the attacker tactic to each portion of the session, i.e., unveiling the sequence of the attacker's goals. This creates a unique attack fingerprint. We demonstrate LogPrécis capability to support the analysis of two large datasets containing about 400,000 unique Unix shell attacks recorded in a 2-year-long honeypot deployment. LogPrécis reduces the analysis to about 3,000 unique fingerprints. Such abstraction lets us better understand attacks, extract attack prototypes, detect novelties, and track families and mutations. Overall, LogPrécis, released as open source, demonstrates the potential of adopting LMs for security analysis and paves the way for better and more responsive defence against cyberattacks.

Sound-squatting is a squatting technique that exploits similarities in word pronunciation to trick users into accessing malicious resources. It is an understudied threat that has gained traction with the popularity of smart speakers and audio-only content, such as podcasts. The picture gets even more complex when multiple languages are involved. We here introduce X-squatter, a multi- and cross-language AI-based system that relies on a Transformer Neural Network for generating high-quality sound-squatting candidates. We illustrate the use of X-squatter by searching for domain name squatting abuse across hundreds of millions of issued TLS certificates, alongside other squatting types. Key findings unveil that approximately 15% of generated sound-squatting candidates have associated TLS certificates, well above the prevalence of other squatting types (7%). Furthermore, we employ X-squatter to assess the potential for abuse in PyPI packages, revealing the existence of hundreds of candidates within a 3-year package history. Notably, our results suggest that the current platform checks cannot handle sound-squatting attacks, calling for better countermeasures. We believe X-squatter uncovers the usage of multilingual sound-squatting phenomena on the Internet and it is a crucial asset for proactive protection against the threat.

IoT plays an important role in cellular networks, and its need for global connectivity is driving the rise of Global IoT Providers. These provide service by aggregating multiple mobile providers through roaming, complicating the understanding of the overall mobile ecosystem. This calls for lightweight monitoring solutions, which are crucial to meet the quality demanded by IoT services, and of automatic means to analyze the data, with the final goal to carry out economic and management activities. This paper provides insights from the study of two commercial, widespread IoT providers. We show how monitoring signaling traffic between mobile networks offers a unique opportunity to understand both the IoT customers’ characteristics and the network functioning. Leveraging clustering, we offer the first data-driven methodology to examine large IoT signaling datasets. By analyzing over 1.3 billion signaling dialogues across two providers, we identify common signaling profiles that depend on the specific IoT vertical, likely misconfigured devices, and sudden changes that indicate potential problems. This provides actionable insights for network management decisions and service improvements, and lays the groundwork for future research on IoT traffic modeling.

Artificial Intelligence (AI) approaches have emerged as powerful tools to improve traffic analysis for network monitoring and management. However, the lack of large labeled datasets and the ever-changing networking scenarios make a fundamental difference compared to other domains where AI is thriving. We believe the ability to transfer the specific knowledge acquired in one network (or dataset) to a different network (or dataset) would be fundamental to speed up the adoption of AI-based solutions for traffic analysis and other networking applications (e.g., cybersecurity). We here propose and evaluate different options to transfer the knowledge built from a provider network, owning data and labels, to a customer network that desires to label its traffic but lacks labels. We formulate this problem as a domain adaptation problem that we solve with embedding alignment techniques and canonical transfer learning approaches. We present a thorough experimental analysis to assess the performance considering both supervised (e.g., classification) and unsupervised (e.g., novelty detection) downstream tasks related to darknet and honeypot traffic. Our experiments show the proper transfer techniques to use the models obtained from a network in a different network. We believe our contribution opens new opportunities and business models where network providers can successfully share their knowledge and AI models with customers.

Recently, Google proposed the Topics API framework as a privacy-friendly alternative for behavioural advertising as a possible solution to balance user’s privacy and advertisement effectiveness. Using the Topics API, the browser builds a user profile based on navigation history, which advertisers can access. The Topics API aim at becoming the new standard for behavioural advertising, thus it is necessary to fully understand its operation and find possible limitations. In this article, we evaluate the robustness of the Topics API to a re-identification attack. To build a user profile, we suppose an attacker accumulates over time the topics a user exposes to different websites. The attacker later re-identifies the same user matching the profiles of their audience. We leverage real traffic traces and realistic population models, and we present increasingly powerful attack threats. We find that the Topics API mitigates but cannot prevent re-identification from taking place, as there is a sizeable chance that a user’s profile remains unique within a website’s audience and the attacker successfully matches it with the profile of the same user on a second website. Depending on environmental factors, the probability of correct re-identification can reach 50%, considering a pool of 1,000 users. We offer the code and data we use in this work to stimulate further studies and the tuning of the Topic API parameters. 1

In the era of Big Data, effective data reduction through feature selection is of paramount importance for machine learning. This paper presents GLEm-Net (Grouped Lasso with Embeddings Network), a novel neural framework that seamlessly processes both categorical and numerical features to reduce the dimensionality of data while retaining as much information as possible. By integrating embedding layers, GLEm-Net effectively manages categorical features with high cardinality and compresses their information in a less dimensional space. By using a grouped Lasso penalty function in its architecture, GLEm-Net simultaneously processes categorical and numerical data, efficiently reducing high-dimensional data while preserving the essential information. We test GLEm-Net with a real-world application in an industrial environment where 6 million records exist and each is described by a mixture of 19 numerical and 7 categorical features with a strong class imbalance. A comparative analysis using state-of-the-art methods shows that despite the difficulty of building a high-performance model, GLEm-Net outperforms the other methods in both feature selection and classification, with a better balance in the selection of both numerical and categorical features.

Web tracking through third-party cookies is considered a threat to users' privacy and is supposed to be abandoned in the near future. Recently, Google proposed the Topics API framework as a privacy-friendly alternative for behavioural advertising. Using this approach, the browser builds a user profile based on navigation history, which advertisers can access. The Topics API has the possibility of becoming the new standard for behavioural advertising, thus it is necessary to fully understand its operation and find possible limitations. This paper evaluates the robustness of the Topics API to a re-identification attack where an attacker reconstructs the user profile by accumulating user's exposed topics over time to later re-identify the same user on a different website. Using real traffic traces and realistic population models, we find that the Topics API mitigates but cannot prevent re-identification to take place, as there is a sizeable chance that a user's profile is unique within a website's audience. Consequently, the probability of correct re-identification can reach 15-17%, considering a pool of 1,000 users. We offer the code and data we use in this work to stimulate further studies and the tuning of the Topic API parameters.

Darknets collect unsolicited traffic reaching unused address spaces. They provide insights into malicious activities, such as the rise of botnets and DDoS attacks. However, darknets provide a shallow view, as traffic is never responded. Here we quantify how their visibility increases by responding to traffic with interactive responders with increasing levels of interaction. We consider four deployments: Darknets, simple, vertical bound to specific ports, and, a honeypot that responds to all protocols on any port. We contrast these alternatives by analyzing the traffic attracted by each deployment and characterizing how traffic changes throughout the responder lifecycle on the darknet. We show that the deployment of responders increases the value of darknet data by revealing patterns that would otherwise be unobservable. We measure Side-Scan phenomena where once a host starts responding, it attracts traffic to other ports and neighboring addresses. uncovers attacks that darknets and would not observe, e.g. large-scale activity on non-standard ports. And we observe how quickly senders can identify and attack new responders. The “enlightened” part of a darknet brings several benefits and offers opportunities to increase the visibility of sender patterns. This information gain is worth taking advantage of, and we, therefore, recommend that organizations consider this option.

URLs play an essential role on the Internet, allowing access to Web resources. Automatically generating URLs is helpful in various tasks, such as application debugging, API testing, and blocklist creation for security applications. Current testing suites deeply embed experts’ domain knowledge to generate suitable URLs, resulting in an ad-hoc solution for each given application. These tools thus require heavy manual intervention, with the expensive coding of rules that are hard to maintain. We here introduce URLGEN, a system that uses Generative Adversarial Networks (GANs) to tackle the automatic URL generation problem. URLGEN is designed for web API testing and generates URL samples for an application without any system expertise, complementing the existing tools. It leverages Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) architectures, augmented by an embedding layer that simplifies the URL learning and generation process. We show that URLGEN learns to generate new valid URLs from samples of real URLs without requiring any domain knowledge and following a purely data-driven approach. We compare the GAN architecture of URLGEN against other design options and show that the LSTM architecture can better capture the correlation among URL characters, outperforming previously proposed solutions. Finally, we show that the URLGEN approach can be extended to other scenarios, which we illustrate with two use cases, i.e., cybersquatting domain prediction and URL classification.

Darknets are probes listening to traffic reaching IP addresses that host no services. Traffic reaching a darknet results from the actions of internet scanners, botnets, and possibly misconfigured hosts. Such peculiar nature of the darknet traffic makes darknets a valuable instrument to discover malicious online activities, e.g., identifying coordinated actions performed by bots or scanners. However, the massive amount of packets and sources that darknets observe makes it hard to extract meaningful insights, calling for scalable tools to automatically identify and group sources that share similar behaviour. We here present i-DarkVec, a methodology to learn meaningful representations of Darknet traffic. i-DarkVec leverages Natural Language Processing techniques (e.g., Word2Vec) to capture the co-occurrence patterns that emerge when scanners or bots launch coordinated actions. As in NLP problems, the embeddings learned with i-DarkVec enable several new machine learning tasks on the darknet traffic, such as identifying clusters of senders engaged in similar activities. We extensively test i-DarkVec and explore its design space in a case study using real darknets. We show that with a proper definition of services , the learned embeddings can be used to (i) solve the classification problem to associate unknown sources’ IP addresses to the correct classes of coordinated actors and (ii) automatically identify clusters of previously unknown sources performing similar attacks and scans, easing the security analyst’s job. i-DarkVec leverages a novel incremental embedding learning approach that is scalable and robust to traffic changes, making it applicable to dynamic and large-scale scenarios.

Privacy on the Internet has become a priority, and several efforts have been devoted to limit the leakage of personal information. Domain names, both in the TLS Client Hello and DNS traffic, are among the last pieces of information still visible to an observer in the network. The Encrypted Client Hello extension for TLS, DNS over HTTPS or over QUIC protocols aim to further increase network confidentiality by encrypting the domain names of the visited servers. In this article, we check whether an attacker able to passively observe the traffic of users could still recover the domain name of websites they visit even if names are encrypted. By relying on large-scale network traces, we show that simplistic features and off-the-shelf machine learning models are sufficient to achieve surprisingly high precision and recall when recovering encrypted domain names. We consider three attack scenarios, i.e., recovering the per-flow name, rebuilding the set of visited websites by a user, and checking which users visit a given target website. We next evaluate the efficacy of padding-based mitigation, finding that all three attacks are still effective, despite resources wasted with padding. We conclude that current proposals for domain encryption may produce a false sense of privacy, and more robust techniques should be envisioned to offer protection to end users.

Transaction Monitoring is one of the main labor-intensive tasks of anti-financial crime and it requires to scrutinise billions of transactions per month against possible crimes. The first step in the process is the correct identification of the involved parties. This foundational step defines the focal entities on which transaction monitoring algorithms rely to spot suspicious events. Unfortunately, the loose syntax of protocols and the free text fields of inter-banking communications make party disambiguation particularly challenging. The first step of a fully automated data-driven strategy is thus the detection of the actual entity owning or using a given account.In this paper, we leverage data-driven techniques to identify and disambiguate the owners of accounts involved in cross-border international transactions when a Financial Institution only knows a minority fraction of such parties as its own customers. For this, we propose a data science pipeline relying on hierarchical clustering to capture similarities among names of parties involved in actual transactions. We test and tune the proposed approach using a large, real-world, multi-language, proprietary dataset of actual international transactions. Our highly parallel implementation completes the identification of parties that share an account and identifies all accounts owned by a party with f-score higher than 0.8.

Satellite Communication (SatCom) offers internet connectivity where traditional infrastructures are too expensive to deploy. When using satellites in a geostationary orbit, the distance from Earth forces a round trip time higher than 550 ms. Coupled with the limited and shared capacity of the physical link, this poses a challenge to the traditional internet access quality we are used to.

“Roam like Home” is the initiative of the European Commission (EC) to end the levy of extra charges when roaming within the European region. As a result, people can use data services more freely across Europe. However, the implications of roaming solutions on network performance have not been carefully examined yet. This paper provides an in-depth characterization of the implications of international data roaming within Europe. We build a unique roaming measurement platform using 16 different mobile networks deployed in six countries across Europe. Using this platform, we measure different aspects of international roaming in 4G networks in Europe, including mobile network configuration, performance characteristics, and quality of experience. We find that operators adopt a common approach to implement roaming called Home-routed roaming (HR). This results in additional latency penalties of 60 ms or more, depending on geographical distance. This leads to worse browsing performance, with an increase in the metrics related to Quality of Experience (QoE) of users (Page Load time and Speed Index) in the order of 15-20 percent. We further analyze in isolation the impact of latency on QoE metrics and find that the penalty imposed by HR leads to a degradation on QoE metrics up to 150 percent in case of intercontinental roaming.

To protect user privacy, legislators have regulated the use of tracking technologies, mandating the acquisition of users’ consent before collecting data. As a result, websites started showing more and more consent management modules–i.e., Consent Banners–the visitors have to interact with to access the website content. Since these banners change the content the browser loads, they challenge web measurement collection, primarily to monitor the extent of tracking technologies, but also to measure web performance. If not correctly handled, Consent Banners prevent crawlers from observing the actual content of the websites. In this paper, we present a comprehensive measurement campaign focusing on popular websites in Europe and the US, visiting both landing and internal pages from different countries around the world. We engineer Priv-Accept , a Web crawler able to accept the Consent Banners, as most users would do in practice. It lets us compare how webpages change before and after accepting such policies, if present. Our results show that all measurements performed ignoring the Consent Banners offer a biased and partial view of the Web. After accepting the privacy policies, web tracking is far more pervasive, and webpages are larger and slower to load.

Darknets are passive probes listening to traffic reaching IP addresses that host no services. Traffic reaching them is unsolicited by nature and often induced by scanners, malicious senders and misconfigured hosts. Its peculiar nature makes it a valuable source of information to learn about malicious activities. However, the massive amount of packets and sources that reach darknets makes it hard to extract meaningful insights. In particular, multiple senders contact the darknet while performing similar and coordinated tasks, which are often commanded by common controllers (botnets, crawlers, etc.). How to automatically identify and group those senders that share similar behaviors remains an open problem.

Abstract Fueled by advertising companies’ need of accurately tracking users and their online habits, web fingerprinting practice has grown in recent years, with severe implications for users’ privacy. In this paper, we design, engineer and evaluate a methodology which combines the analysis of JavaScript code and machine learning for the automatic detection of web fingerprinters. We apply our methodology on a dataset of more than 400, 000 JavaScript files accessed by about 1, 000 volunteers during a one-month long experiment to observe adoption of fingerprinting in a real scenario. We compare approaches based on both static and dynamic code analysis to automatically detect fingerprinters and show they provide different angles complementing each other. This demonstrates that studies based on either static or dynamic code analysis provide partial view on actual fingerprinting usage in the web. To the best of our knowledge we are the first to perform this comparison with respect to fingerprinting. Our approach achieves 94% accuracy in small decision time. With this we spot more than 840 fingerprinting services, of which 695 are unknown to popular tracker blockers. These include new actual trackers as well as services which use fingerprinting for purposes other than tracking, such as anti-fraud and bot recognition.

Packet measurements at scale are essential for several applications, such as cyber-security, accounting and troubleshooting. They, however, threaten users’ privacy by exposing sensitive information. Anonymization has been the answer to this challenge, i.e., replacing sensitive information with obfuscated copies. Anonymization of packet traces, however, comes with some challenges and drawbacks. First, it reduces the value of data. Second, it requires to consider diverse protocols because information may leak from many non-encrypted fields. Third, it must be performed at high speeds directly at the monitor, to prevent private data from leaking, calling for real-time solutions. We present$\alpha $-MON, a flexible tool for privacy-preserving packet monitoring. It replicates input packet streams to different consumers while anonymizing protocol fields according to flexible policies that cover all protocol layers. Beside classic anonymization mechanisms such as IP address obfuscation,$\alpha $-MON supports${z}$-anonymization, a novel solution to obfuscate rare values that can be uniquely traced back to limited sets of users. Differently from classic anonymization approaches,z-anonymityworks on a streaming fashion, with zero delay, operating at high-speed links on a packet-by-packet basis. We quantify the impact ofz-anonymityon traffic measurements, finding that it introduces minimal error when it comes to finding heavy-hitter services. We evaluate$\alpha $-MON performance using packet traces collected from an ISP network and show that it achieves a sustainable rate of 40 Gbit/s on a Commercial Off-the Shelf server.$\alpha $-MON is available to the community as an open-source project.

With the advent of big data and the birth of the data markets that sell personal information, individuals' privacy is of utmost importance. The classical response is anonymization, i.e., sanitizing the information that can directly or indirectly allow users' re-identification. The most popular solution in the literature is the k-anonymity. However, it is hard to achieve k-anonymity on a continuous stream of data, as well as when the number of dimensions becomes high.In this paper, we propose a novel anonymization property called z-anonymity. Differently from k-anonymity, it can be achieved with zero-delay on data streams and it is well suited for high dimensional data. The idea at the base of z-anonymity is to release an attribute (an atomic information) about a user only if at least z - 1 other users have presented the same attribute in a past time window. z-anonymity is weaker than k-anonymity since it does not work on the combinations of attributes, but treats them individually. In this paper, we present a probabilistic framework to map the z-anonymity into the k-anonymity property. Our results show that a proper choice of the z-anonymity parameters allows the data curator to likely obtain a k-anonymized dataset, with a precisely measurable probability. We also evaluate a real use case, in which we consider the website visits of a population of users and show that z-anonymity can work in practice for obtaining the k-anonymity too.

With the shift toward a Mobility-as-a-Service paradigm, electric scooter sharing systems are becoming a popular transportation mean in cities. Given their novelty, we lack of consolidated approaches to study and compare different system design options. In this work, we propose a simulation approach that leverages open data to create a demand model that captures and generalises the usage of this transportation mean in a city. This calls for ingenuity to deal with coarse open data granularity. In particular, we create a flexible, data-driven demand model by using modulated Poisson processes for temporal estimation, and Kernel Density Estimation (KDE) for spatial estimation. We next use this demand model alongside a configurable e-scooter sharing simulator to compare performance of different electric scooter sharing design options, such as the impact of the number of scooters and the cost of managing their charging. We focus on the municipalities of Minneapolis and Louisville which provide large scale open data about e-scooter sharing rides. Our approach let researchers, municipalities and scooter sharing providers to follow a data driven approach to compare and improve the design of e-scooter sharing system in smart cities.

The Internet and the way people use it are constantly changing. Knowing traffic is crucial for operating the network, understanding users' needs, and ultimately improving applications. Here, we provide an in-depth longitudinal view of Internet traffic during 5 years (from 2013 to 2017). We take the point of the view of a national-wide ISP and analyze rich flow-level measurements to pinpoint and quantify changes. We observe the traffic, both from a point of view of users and services. We show that an ordinary broadband subscriber downloaded in 2017 more than twice as much as they used to do 5 years before. Bandwidth hungry video services drove this change at the beginning, while recently social messaging applications contribute to increase of data consumption. We study how protocols and service infrastructures evolve over time, highlighting events that may challenge traffic management policies. In the rush to bring servers closer and closer to users, we witness the birth of the sub-millisecond Internet, with caches located directly at ISP edges. The picture we take shows a lively Internet that always evolves and suddenly changes. To support new analyses, we make anonymized data available at https://smartdata.polito.it/five-years-at-the-edge/.

Darknets are sets of IP addresses that are advertised but do not host any client or server. By passively recording the incoming packets, they assist network monitoring activities. Since packets they receive are unsolicited by definition, darknets help to spot misconfigurations as well as important security events, such as the appearance and spread of botnets, DDoS attacks using spoofed IP address, etc. A number of organizations worldwide deploys darknets, ranging from a few dozens of IP addresses to large /8 networks. We here investigate how similar is the visibility of different darknets. By relying on traffic from three darknets deployed in different contintents, we evaluate their exposure in terms of observed events given their allocated IP addresses. The latter is particularly relevant considering the shortage of IPv4 addresses on the Internet. Our results suggest that some well-known facts about darknet visibility seem invariant across deployments, such as the most commonly contacted ports. However, size and location matter. We find significant differences in the observed traffic from darknets deployed in different IP ranges as well as according to the size of the IP range allocated for the monitoring.

Measuring and characterizing web page performance is a challenging task. When it comes to the mobile world, the highly varying technology characteristics coupled with the opaque network configuration make it even more difficult. Aiming at reproducibility, we present a large scale empirical study of web page performance collected in eleven commercial mobile networks spanning four countries. By digging into measurement from nearly two million web browsing sessions, we shed light on the impact of different web protocols, browsers, and mobile technologies on the web performance. We find that the impact of mobile broadband access is sizeable. For example, the median page load time using mobile broadband increases by a third compared to wired access. Mobility clearly stresses the system, with handover causing the most evident performance penalties. Contrariwise, our measurements show that the adoption of HTTP/2 and QUIC has practically negligible impact. To understand the intertwining of all parameters, we adopt state-of-the-art statistical methods to identify the significance of different factors on the web performance. Our analysis confirms the importance of access technology and mobility context as well as webpage composition and browser. Our work highlights the importance of large-scale measurements. Even with our controlled setup, the complexity of the mobile web ecosystem is challenging to untangle. For this, we are releasing the dataset as open data for validation and further research.

Abstract Personalized advertisement has changed the web. It lets websites monetize the content they offer. The downside is the continuous collection of personal information with significant threats to personal privacy. In 2002, the European Union (EU) introduced a first set of regulations on the use of online tracking technologies. It aimed, among other things, to make online tracking mechanisms explicit to increase privacy awareness among users. Amended in 2009, the EU Directive mandates websites to ask for informed consent before using any kind of profiling technology, e.g., cookies. Since 2013, the ePrivacy Directive became mandatory, and each EU Member State transposed it in national legislation. Since then, most of European websites embed a “Cookie Bar”, the most visible effect of the regulation. In this paper, we run a large-scale measurement campaign to check the current implementation status of the EU cookie directive. For this, we use CookieCheck, a simple tool to automatically verify legislation violations. Results depict a shady picture: 49 % of websites do not respect the Directive and install profiling cookies before any user’s consent is given. Beside presenting a detailed picture, this paper casts lights on the difficulty of legislator attempts to regulate the troubled marriage between ad-supported web services and their users. In this picture, online privacy seems to be continuously at stake, and it is hard to reach transparency.

In this paper, we study the design of a free floating car sharing system based on electric vehicles. We rely on data about millions of rentals of a free floating car sharing operator based on internal combustion engine cars that we recorded in four cities. We characterize the nature of rentals, highlighting the non-stationary, and highly dynamic nature of usage patterns. Building on this data, we develop a discrete-event trace-driven simulator to study the usage of a hypothetical electric car sharing system. We use it to study the charging station placement problem, modeling different return policies, car battery charge and discharge due to trips, and the stochastic behavior of customers for plugging a car to a pole. Our data-driven approach helps car sharing providers to gauge the impact of different design solutions. Our simulations show that it is preferred to place charging stations within popular parking areas where cars are parked for short time (e.g., downtown). By smartly placing charging stations in just 8% of city zones, no trip ends with a discharged battery, i.e., all trips are feasible. Customers shall collaborate by bringing the car to a charging station when the battery level goes below a minimum threshold. This may reroute the customer to a different destination zone than the desired one; however, this happens in less than 10% of all trips.

Network Traffic Monitoring and Analysis (NTMA) represents a key component for network management, especially to guarantee the correct operation of large-scale networks such as the Internet. As the complexity of Internet services and the volume of traffic continue to increase, it becomes difficult to design scalable NTMA applications. Applications such as traffic classification and policing require real-time and scalable approaches. Anomaly detection and security mechanisms require to quickly identify and react to unpredictable events while processing millions of heterogeneous events. At last, the system has to collect, store, and process massive sets of historical data for post-mortem analysis. Those are precisely the challenges faced by general big data approaches: Volume, Velocity, Variety, and Veracity. This survey brings together NTMA and big data. We catalog previous work on NTMA that adopt big data approaches to understand to what extent the potential of big data is being explored in NTMA. This survey mainly focuses on approaches and technologies to manage the big NTMA data, additionally briefly discussing big data analytics (e.g., machine learning) for the sake of NTMA. Finally, we provide guidelines for future work, discussing lessons learned, and research directions.

In this paper, we present longitudinal exploration for network traffic analysis (LENTA), a system that supports the network analysts in the identification of traffic generated by services and applications running on the Web. In the case of URLs observed in operative network, LENTA simplifies the analyst's job by letting her observe few hundreds of clusters instead of the original hundred thousands of single URLs. We implement a self-learning methodology, where the system grows its knowledge, which is used in turn to automatically associate traffic to previously observed services, and identify new traffic generated by possibly suspicious applications. This approach lets the analysts easily observe changes in network traffic, identify new services, and unexpected activities. We follow a data-driven approach and run LENTA on traces collected both in ISP networks and directly on hosts via proxies. We analyze traffic in batches of 24-h worth of traffic. Big data solutions are used to enable horizontal scalability and meet performance requirements. We show that LENTA allows the analyst to clearly understand which services are running on their network, possibly highlighting malicious traffic and changes over time, greatly simplifying the view and understanding of the network traffic.

Clustering has become an important means to analyze large datasets when labeled data is not available. The volume of data and its variety however challenge classical clustering algorithms, with density-based ones suffering from severe scalability issues.In this paper, we propose a way to perform density-based clustering efficiently by exploiting the horizontal scalability offered by big data solution such as Apache Spark. We are motivated by recent techniques for Internet monitoring that rely on clustering to group similar events and spot anomalies. We focus specifically on textual data, such as URLs or server logs. Computing the distance between points, here represented as strings, becomes a major issue. Indeed, when datasets become large, most of density-based clustering algorithms are bottlenecked by the computation of all the distances between any pairs of elements. To overcome this, we propose to decouple the distance computation, easily amenable to parallelization, from the algorithm execution. By using this approach, we can easily exploit the benefits of distributed platforms like Apache Spark or MapReduce. A faster execution of the algorithms is thus guaranteed, together with more flexibility in the choice of the clustering method.We make both the code and the dataset publicly available, to both guarantee the repeatability of the experiments, and possibly offering a new benchmark dataset.

In April 2018, under the auspices of the POR-FESR 2014-2020 program of Italian Piedmont Region, the Turin's Centre on High-Performance Computing for Artificial Intelligence (HPC4AI) was funded with a capital investment of 4.5M€ and it began its deployment. HPC4AI aims to facilitate scientific research and engineering in the areas of Artificial Intelligence and Big Data Analytics. HPC4AI will specifically focus on methods for the on-demand provisioning of AI and BDA Cloud services to the regional and national industrial community, which includes the large regional ecosystem of Small-Medium Enterprises (SMEs) active in many different sectors such as automotive, aerospace, mechatronics, manufacturing, health and agrifood.

The Internet and the way people use it are constantly changing. Knowing traffic is crucial for operating the network, understanding users' need, and ultimately improving applications. Here, we provide an in-depth longitudinal view of Internet traffic in the last 5 years (from 2013 to 2017). We take the point of the view of a national-wide ISP and analyze flow-level rich measurements to pinpoint and quantify trends. We evaluate the providers' costs in terms of traffic consumption by users and services. We show that an ordinary broadband subscriber nowadays downloads more than twice as much as they used to do 5 years ago. Bandwidth hungry video services drive this change, while social messaging applications boom (and vanish) at incredible pace. We study how protocols and service infrastructures evolve over time, highlighting unpredictable events that may hamper traffic management policies. In the rush to bring servers closer and closer to users, we witness the birth of the sub-millisecond Internet, with caches located directly at ISP edges. The picture we take shows a lively Internet that always evolves and suddenly changes.

Predictive maintenance is an ever-growing area of interest, spanning different fields and approaches. In the automotive industry faulty behaviors of the oxygen sensor are a key challenge to address. This paper presents OxyClog, a data-driven framework that, given a large number of time series collected from a vehicle's ECU (engine control unit), builds a model to predict if the oxygen sensor is currently unclogged, almost clogged (since the clogging of the sensor happens gradually), or clogged. OxyClog is characterized by a tailored preprocessing, which includes a custom and interpretable feature selection algorithm, along with a summarization strategy to transform a time-dependent problem into a time-independent one. Furthermore, a semi-supervised labeling methodology has been devised to use different data sources with different characteristics to define meaningful clogging labels. OxyClog integrates state-of-the-art classification algorithms - both interpretable and non-interpretable - to process real ECU data with good prediction performance.

"Roam like Home" is the initiative of the European Commission (EC) to end the levy of extra charges when roaming within the European region. As a result, people are able to use data services more freely across Europe. However, the implications roaming solutions have on performance have not been carefully examined. This paper provides an in-depth characterization of the implications of international data roaming within Europe. We build a unique roaming measurement platform using 16 different mobile networks deployed in six countries across Europe. Using this platform, we measure different aspects of international roaming in 3G and 4G networks, including mobile network configuration, performance characteristics, and content discrimination. We find that operators adopt common approaches to implementing roaming, resulting in additional latency penalties of ∼60 ms or more, depending on geographical distance. Considering content accessibility, roaming poses additional constraints that leads to only minimal deviations when accessing content in the original country. However, geographical restrictions in the visited country make the picture more complicated and less intuitive.

Car sharing is a popular means of transport in smart cities. The free floating paradigm lets the customers autonomously pick and drop available cars freely, within city limits. In this work we study the different policies when designing an electric Free Floating Car Sharing (FFCS) system. This system has the need to guarantee battery charge, a time-consuming operation, for which charging stations availability becomes a key factor for the sustainability of the whole system. We harvest the data of an already operative FFCS provider, and extract information about actual users' driving patterns. We implement a trace driven simulator to replay collected users' trips and simulate car batteries consumption for different design parameters. In this work, we limit the study to a single city, Turin (Italy), where we leverage actual trips registered over 2 months. We analyse and discuss several system design alternatives: the number of charging stations, their placement, and when to force users to return cars for charge. We identify regimes where cars never discharge and users can freely drop cars anywhere, albeit they are rarely rerouted to a charging station, possibly located in a nearby area to their original destination. Surprisingly, our data shows that even few charging stations (15 or more, i.e., 6% of city areas) guarantees the system to work almost autonomously, making thus possible free floating car sharing a feasible solution with electric cars.

Cloud and network analytics can harness the immense stream of operational data from clouds and networks, and can perform analytics processing to improve reliability, automated configuration, performance, and optimized network management in general. In this area, we have witnessed a growing trend towards using statistical analysis and machine learning techniques to improve operations and management of IT systems and networks.

Software defined network (SDN) has enabled consistent and programmable management in computer networks. However, the explosion of cloud services and content delivery networks (CDNs)-coupled with the momentum of encryption-challenges the simple per-flow management and calls for a more comprehensive approach for managing Web traffic. We propose a new approach based on a “per service” management concept, which allows to identify and prioritize all traffic of important Web services, while segregating others, even if they are running on the same cloud platform, or served by the same CDN. We design and evaluate AWESoME, automatic Web service manager, a novel SDN application to address the above problem. On the one hand, it leverages big data algorithms to automatically build models describing the traffic of thousands of Web services. On the other hand, it uses the models to install rules in SDN switches to steer all flows related to the originating services. Using traffic traces from volunteers and operational networks, we provide extensive experimental results to show that AWESoME associates flows to the corresponding Web service in real-time and with high accuracy. AWESoME introduces a negligible load on the SDN controller and installs a limited number of rules on switches, hence scaling well in realistic deployments. Finally, for easy reproducibility, we release ground truth traces and scripts implementing AWESoME core components.

Understanding how people interact with the web is key for a variety of applications, e.g., from the design of effective web pages to the definition of successful online marketing campaigns. Browsing behavior has been traditionally represented and studied by means of clickstreams , i.e., graphs whose vertices are web pages, and edges are the paths followed by users. Obtaining large and representative data to extract clickstreams is, however, challenging. The evolution of the web questions whether browsing behavior is changing and, by consequence, whether properties of clickstreams are changing. This article presents a longitudinal study of clickstreams from 2013 to 2016. We evaluate an anonymized dataset of HTTP traces captured in a large ISP, where thousands of households are connected. We first propose a methodology to identify actual URLs requested by users from the massive set of requests automatically fired by browsers when rendering web pages. Then, we characterize web usage patterns and clickstreams, taking into account both the temporal evolution and the impact of the device used to explore the web. Our analyses precisely quantify various aspects of clickstreams and uncover interesting patterns, such as the typical short paths followed by people while navigating the web, the fast increasing trend in browsing from mobile devices, and the different roles of search engines and social networks in promoting content. Finally, we contribute a dataset of anonymized clickstreams to the community to foster new studies.1

The DNS is a fundamental service that has been repeatedly attacked and abused. DNS manipulation is a prominent case: Recursive DNS resolvers are deployed to explicitly return manipulated answers to users' queries. While DNS manipulation is used for legitimate reasons too (e.g., parental control), rogue DNS resolvers support malicious activities, such as malware and viruses, exposing users to phishing and content injection. We introduce REMeDy, a system that assists operators to identify the use of rogue DNS resolvers in their networks. REMeDy is a completely automatic and parameter-free system that evaluates the consistency of responses across the resolvers active in the network. It operates by passively analyzing DNS traffic and, as such, requires no active probing of third-party servers. REMeDy is able to detect resolvers that manipulate answers, including resolvers that affect unpopular domains. We validate REMeDy using large-scale DNS traces collected in ISP networks where more than 100 resolvers are regularly used by customers. REMeDy automatically identifies regular resolvers, and pinpoint manipulated responses. Among those, we identify both legitimate services that offer additional protection to clients, and resolvers under the control of malwares that steer traffic with likely malicious goals.

In modern web, users contact lots of services, identified by the domain name of the server. The temporal sequence and transitions of visited domains form a trajectory of the user on the web. In this work, we analyze 4 weeks of such trajectories, extracted from logs collected in our university network, and mine them via big data and machine learning methodologies to extract the interests of users. Our goal is to create a model of such trajectories and find similarities so to observe peculiarity of users' browsing. Thanks to the model, we propose a methodology to automatically group together the trajectories of single users and/or communities into highly descriptive environments which in turn allow the analyst to identify the topic of interest. We propose an automatic way to highlight differences in terms of popularity and content of environments. Lastly, we analyze the transition among environments, showing how people in smaller communities, e.g., in the same department, have a much more homogeneous behaviour than people at large, e.g., in the university.

We consider the practical problem of video surveillance in public transport systems, where security videos are stored onboard, and a central operator occasionally needs to access portions of the recordings. When this happens, the selected video must be uploaded within a deadline, possibly using multiple parallel wireless interfaces. Interfaces have different associated costs, related to tariffs charged by Mobile Network Operators (MNOs), energy consumption, data quotas, system load. Our goal is to choose which interfaces to use, and when, so as to minimize the cost of the upload while meeting the deadline, despite the unknown short-term variations in throughput. To achieve this goal, we first collect real traces of mobile uploads from vehicles for different MNOs. Examination of these traces confirms the unpredictability of the short-term throughput of wireless connections, and motivates the adoption of adaptive schedulers with limited a-priori knowledge of the system status. To effectively solve our problem, we devised a family of adaptive algorithms, that we thoroughly evaluated using a trace-driven approach. Results show that our adaptive approach can effectively leverage the fundamental tradeoff between the total cost and the delivery time of content upload, despite unknown short-term variations in throughput.

Open experimentation with operational Mobile Broadband (MBB) networks in the wild is currently a fundamental requirement of the research community in its endeavor to address the need of innovative solutions for mobile communications. Even more, there is a strong need for objective data about stability and performance of MBB (e.g., 3G/4G) networks, and for tools that rigorously and scientifically assess their status. In this paper, we introduce the MONROE measurement platform: an open access and flexible hardware-based platform for measurements and custom experimentation on operational MBB networks. The MONROE platform enables accurate, realistic and meaningful assessment of the performance and reliability of 11 MBB networks in Europe. We report on our experience designing, implementing and testing the solution we propose for the platform. We detail the challenges we overcame while building and testing the MONROE testbed and argue our design and implementation choices accordingly. We describe and exemplify the capabilities of the platform and the wide variety of experiments that external users already perform using the system.

One of the limits of web content discovery tools, let them be recommender systems or content curation tools such as social rating, social bookmarking and other social media, is the scarcity of user input (e.g. rate, submit, share). This problem is even worse in the case of what we call communities of a place: people who study, live or work at the same place. Such people often share common interests but either do not know each other or fail to actively engage in submitting and relaying information. In this paper, we investigate the feasibility of using the aggregated clicks of entire communities of users to passively emulate a content curation service a la Reddit. To this end, we prototype and deploy WeBrowse, a content curation service based on the processing of raw HTTP logs. Evaluation based on our deployments demonstrates feasibility at scale while respecting user privacy. The majority of WeBrowse's users welcome the quality of content it promotes.

The large amount of space offered by personal cloud storage services (e.g., Dropbox and OneDrive), together with the possibility of synchronizing devices seamlessly, keep attracting customers to the cloud. Despite the high public interest, little information about system design and actual implications on performance is available when selecting a cloud storage service. Systematic benchmarks to assist in comparing services and understanding the effects of design choices are still lacking. This paper proposes a methodology to understand and benchmark personal cloud storage services. Our methodology unveils their architecture and capabilities. Moreover, by means of repeatable and customizable tests, it allows the measurement of performance metrics under different workloads. The effectiveness of the methodology is shown in a case study in which 11 services are compared under the same conditions. Our case study reveals interesting differences in design choices. Their implications are assessed in a series of benchmarks. Results show no clear winner, with all services having potential for improving performance. In some scenarios, the synchronization of the same files can take 20 times longer. In other cases, we observe a wastage of twice as much network capacity, questioning the design of some services. Our methodology and results are thus useful both as benchmarks and as guidelines for system design.

HTTP(S) has become the main means to access the Internet. The web is a tangle, with (i) multiple services and applications co-located on the same infrastructure and (ii) several websites, services and applications embedding objects from CDN, ads and tracking platforms. Traditional solutions for traffic classification and metering fall short in providing visibility in users' activities. Service providers and corporate network administrators are left with huge amounts of measurements, which cannot immediately reveal the real impact of each web service on the network. Such visibility is key to dimension the network, charge users and policy traffic. This paper introduces the Web Helper Accounting Tool (WHAT), a system to uncover the overall traffic produced by specific web services. WHAT combines big data and machine learning approaches to process large volumes of network flow measurements and learn how to group traffic due to pre-defined services of interest. Our evaluation demonstrates WHAT effectiveness in enabling accurate accounting of the traffic associated to each service. WHAT illustrates the power of machine learning when applied to large datasets of network measurements, and allows network administrators to regain the lost visibility on network usage.

The identification of the services that generate traffic is crucial for ISPs and companies to plan and monitor the network. The widespread deployment of encryption and the convergence of the web services towards HTTP/HTTPS challenge traditional classification techniques. Algorithms to classify traffic are left with little information, such as server IP addresses, flow characteristics and queries performed at the DNS. Moreover, due to the usage of Content Delivery Networks and cloud infrastructure, it is unclear whether such coarse metadata is sufficient to differentiate the traffic. This paper studies to what extent basic information visible at flow-level measurements is useful for traffic classification on the web. By analyzing a large dataset of flow measurements, we quantify how often the same server IP address is used by different services, and how services use hostnames. Our results show that a very simple classifier that relies only on server IP addresses and on lists of hostnames can distinguish up to 55% of the traffic volume. Yet, collisions of names and addresses are common among popular services, calling for more ingenuity. This paper is a preliminary step in the evaluation of classification algorithms that are suitable for the modern Internet, where only minimal metadata collection will be possible in the network.

Detecting explicit user actions, i.e., requests for web pages such as hyper-link clicks, from passive traces is fundamental for many applications, such as network forensics or content popularity estimation. Every URL explicitly visited by a user usually triggers further automatic URL requests to obtain all objects that compose the web page. HTTP traces provide a summary of all URLs requested by users, but no information that could be used to separate explicit from automatic requests. Previous works have targeted this problem and ad-hoc heuristics have been proposed. Validation has been typically done using synthetic traces. This paper investigates whether an approach based solely on machine learning can successfully detect user actions from HTTP traces. A machine learning approach would come with many advantages - e.g., it minimizes manual tuning of parameters and can easily adapt to page structure changes. We build both real and synthetic traces to assess the performance and gain insights on the features that bring most advantages in classification. Our results show that machine learning reaches similar or better performance as previous heuristics. Furthermore, we show that models built with machine learning algorithms are robust, presenting consistent performance in different scenarios.

Mobile broadband (MBB) networks underpin numerous vital operations of the society and are arguably becoming the most important piece of the communications infrastructure. In this demo paper, our goal is to showcase the potential of a novel multi-homed MBB platform for measuring, monitoring and assessing the performance of MBB services in an objective manner. Our platform, MONROE, is composed of hundreds of nodes scattered over four European countries and a backend system that collects the measurement results. Through a user-friendly web client, the experimenters can schedule and deploy their experiments. The platform further embeds traffic analysis tools for real-time traffic flow analysis and a powerful visualization tool.

Understanding the behavior of a network from a large scale traffic dataset is a challenging problem. Big data frameworks offer scalable algorithms to extract information from raw data, but often require a sophisticated fine-tuning and a detailed knowledge of machine learning algorithms. To streamline this process, we propose self-learning insightful network analyzer (SeLINA), a generic, self-tuning, simple tool to extract knowledge from network traffic measurements. SeLINA includes different data analytics techniques providing self-learning capabilities to state-of-the-art scalable approaches, jointly with parameter auto-selection to off-load the network expert from parameter tuning. We combine both unsupervised and supervised approaches to mine data with a scalable approach. SeLINA embeds mechanisms to check if the new data fits the model, to detect possible changes in the traffic, and to, possibly automatically, trigger model rebuilding. The result is a system that offers human-readable models of the data with minimal user intervention, supporting domain experts in extracting actionable knowledge and highlighting possibly meaningful interpretations. SeLINA's current implementation runs on Apache Spark. We tested it on large collections of real-world passive network measurements from a nationwide ISP, investigating YouTube, and P2P traffic. The experimental results confirmed the ability of SeLINA to provide insights and detect changes in the data that suggest further analyses.

Malicious activities on the Web are increasingly threatening users in the Internet. Home networks are one of the prime targets of the attackers to host malware, commonly exploited as a stepping stone to further launch a variety of attacks. Due to diversification, existing security solutions often fail to detect malicious activities that remain hidden and pose threats to users' security and privacy. Characterizing behavioral patterns of known malware can help to improve the classification accuracy of threats. More importantly, as different malware might share commonalities, studying the behavior of known malware could help the detection of previously unknown malicious activities. We pose the research question if it is possible to characterize such behavioral patterns analyzing the traffic from known infected clients. We present our quest to discover such characterizations. Results show that commonalities arise but their identification may require some ingenuity. We also present our discovery of malicious activities that were left undetected by commercial IDS.

When browsing, users are consistently tracked by parties whose business builds on the value of collected data. The privacy implications are serious. Consumers and corporates do worry about the information they unknowingly expose to the outside world, and they claim for mechanisms to curb this leakage. Existing countermeasures to web tracking either base on hostname blacklists whose origin is impossible to know and must be continuously updated. This paper presents a novel, unsupervised methodology that leverages application-level traffic logs to automatically detect services running some tracking activity, thus enabling the generation of curated blacklists. The methodology builds on an algorithm that pinpoints pieces of information containing user identifiers exposed in URL queries in HTTP(S) transactions. We validate our algorithm over an artificial dataset obtained by visiting the top 200 most popular websites in the Alexa rank. Results are excellent. Our algorithm identifies 34 new third- party trackers not present in available blacklists. By analyzing the output of our algorithm, some privacy-related interactions emerge. For instance, we observe scenarios clearly hinting to Cookie Matching practice, for which information about users' activity gets shared across several different third-parties.

Malware is a major threat to security and privacy of network users. A huge variety of malware typically spreads over the Internet, evolving every day, and challenging the research community and security practitioners to improve the effectiveness of countermeasures. In this paper, we present a system that automatically extracts patterns of network activity related to a specific malicious event, i.e., a seed. Our system is based on a methodology that correlates network events of hosts normally connected to the Internet over (i) time (i.e., analyzing different samples of traffic from the same host), (ii) space (i.e., correlating patterns across different hosts), and (iii) network layers (e.g., HTTP, DNS, etc.). The result is a Network Connectivity Graph that captures the overall "network behavior" of the seed. That is a focused and enriched representation of the malicious pattern infected hosts exhibit, purified from ordinary network activities and background traffic. We applied our approach on a large dataset collected in a real commercial ISP where the aggregated traffic produced by more than 20,000 households has been monitored. A commercial IDS has been used to complement network data with alerts related to malicious activities. We use such alerts to trigger our processing system. Results shows that the richness of the Network Connectivity Graph provides a much more detailed picture of malicious activities, considerably enhancing our understanding.

HTTP is a popular channel for malware to communicate with malicious servers (e.g., Command & Control, drive-by download, drop-zone), as well as to attack benign servers. By utilizing HTTP requests, malware easily disguises itself under a large amount of benign HTTP traffic. Thus, identifying malicious HTTP activities is challenging. We leverage an insight that cyber criminals are increasingly using dynamic malicious infrastructures with multiple servers to be efficient and anonymous in (i) malware distribution (using redirectors and exploit servers), (ii) control (using C&C servers) and (iii) monetization (using payment servers), and (iv) being robust against server takedowns (using multiple backups for each type of servers). Instead of focusing on detecting individual malicious domains, we propose a complementary approach to identify a group of closely related servers that are potentially involved in the same malware campaign, which we term as Associated Server Herd (ASH). Our solution, SMASH (Systematic Mining of Associated Server Herds), utilizes an unsupervised framework to infer malware ASHs by systematically mining the relations among all servers from multiple dimensions. We build a prototype system of SMASH and evaluate it with traces from a large ISP. The result shows that SMASH successfully infers a large number of previously undetected malicious servers and possible zero-day attacks, with low false positives. We believe the inferred ASHs provide a better global view of the attack campaign that may not be easily captured by detecting only individual servers.

Public IPv4 addresses are a scarce resource. While IPv6 adoption is lagging, Network Address Translation (NAT) technologies have been deployed over the last years to alleviate IPv4 exiguity and their high rental cost. In particular, Carrier-Grade NAT (CGN) is a well known solution to mask a whole ISP network behind a limited amount of public IP addresses, significantly reducing expenses.

Security tools have evolved dramatically in the recent years to combat the increasingly complex nature of attacks, but to be effective these tools need to be configured by experts that understand network protocols thoroughly. In this paper we present FieldHunter, which automatically extracts fields and infers their types; providing this much needed information to the security experts for keeping pace with the increasing rate of new network applications and their underlying protocols. FieldHunter relies on collecting application messages from multiple sessions and then applying statistical correlations is able to infer the types of the fields. These statistical correlations can be between different messages or other associations with meta-data such as message length, client or server IPs. Our system is designed to extract and infer fields from both binary and textual protocols. We evaluated FieldHunter on real network traffic collected in ISP networks from three different continents. FieldHunter was able to extract security relevant fields and infer their nature for well documented network protocols (such as DNS and MSNP) as well as protocols for which the specifications are not publicly available (such as SopCast) and from malware such as (Ramnit).

Peer-to-peer live-streaming (P2P-TV) systems' goal is disseminating real-time video content using peer-to-peer technology. Their performance is driven by the overlay topology, i.e., the virtual topology that peers use to exchange video chunks. Several proposals have been made in the past to optimize it, yet few experimental studies have corroborated results. The aim of this paper is to provide a comprehensive experimental comparison based on PeerStreamer in order to benchmark different strategies for the construction and maintenance of the overlay topology in P2P-TV systems. We present only experimental results in which fully distributed strategies are evaluated in both controlled experiments and the Internet using thousands of peers. Results confirm that the topological properties of the overlay have a deep impact on both user quality of experience and network load. Strategies based solely on random peer selection are greatly outperformed by smart yet simple and actually implementable strategies. The most performing strategy we devise guarantees to deliver almost all chunks to all peers with a playout delay as low as 6 s even when system load approaches 1, and in almost adversarial network scenarios. PeerStreamer is open-source to make results reproducible and allow further research by the community.

This paper considers an approach to identify previously undetected malicious clients in Internet Service Provider (ISP) networks by combining flow classification with a graph-based score propagation method. Our approach represents all HTTP communications between clients and servers as a weighted, near-bipartite graph, where the nodes correspond to the IP addresses of clients and servers while the links are their interconnections, weighted according to the output of a flow-based classifier. We employ a two-phase alternating score propagation algorithm on the graph to identify suspicious clients in a monitored network. Using a symmetrized weighted adjacency matrix as its input, we show that our score propagation algorithm is less vulnerable towards inflating the malicious scores of popular Web servers with high in-degrees compared to the normalization used in PageRank, a widely used graph-based method. Experimental results on a 4-hour network trace collected by a large Internet service provider showed that incorporating flow information into score propagation significantly improves the precision of the algorithm.

The complexity of the Internet has rapidly increased, making it more important and challenging to design scalable network monitoring tools. Network monitoring typically requires rolling data analysis, i.e., continuously and incrementally updating (rolling-over) various reports and statistics over highvolume data streams. In this paper, we describe DBStream, which is an SQL-based system that explicitly supports incremental queries for rolling data analysis. We also present a performance comparison of DBStream with a parallel data processing engine (Spark), showing that, in some scenarios, a single DBStream node can outperform a cluster of ten Spark nodes on rolling network monitoring workloads. Although our performance evaluation is based on network monitoring data, our results can be generalized to other Big Data problems with high volume and velocity.

Increased user concern over security and privacy on the Internet has led to widespread adoption of HTTPS, the secure version of HTTP. HTTPS authenticates the communicating end points and provides confidentiality for the ensuing communication. However, as with any security solution, it does not come for free. HTTPS may introduce overhead in terms of infrastructure costs, communication latency, data usage, and energy consumption. Moreover, given the opaqueness of the encrypted communication, any in-network value added services requiring visibility into application layer content, such as caches and virus scanners, become ineffective.

The authors report an experimental investigation on the measurement of the available bandwidth for the users in gigabit passive optical networks (GPON) and the limitations caused by the Internet protocols, and transfer control protocol (TCP) in particular. We point out that the huge capacity offered by the GPON highlights the enormous differences that can be showed among the available and actually exploitable bandwidth. In fact, while the physical layer capacity can reach value of 100 Mb/s and more, the bandwidth at disposal of the user (i.e. either throughput at transport layer or goodput at application layer) can be much lower when applications and services based on TCP protocol are considered. In the context of service level agreements (SLA) verification, we show how to simultaneously measure throughput and line capacity by offering a method to verify multilayer SLA. We also show how it is possible to better exploit the physical layer capacity by adopting multiple TCP connections avoiding the bottleneck of a single connection.

Clouds and CDNs are systems that tend to separate the content being requested by users from the physical servers capable of serving it. From the network point of view, monitoring and optimizing performance for the traffic they generate are challenging tasks, given that the same resource can be located in multiple places, which can, in turn, change at any time. The first step in understanding cloud and CDN systems is thus the engineering of a monitoring platform. In this paper, we propose a novel solution that combines passive and active measurements and whose workflow has been tailored to specifically characterize the traffic generated by cloud and CDN infrastructures. We validate our platform by performing a longitudinal characterization of the very well known cloud and CDN infrastructure provider Amazon Web Services (AWS). By observing the traffic generated by more than 50 000 Internet users of an Italian Internet Service Provider, we explore the EC2, S3, and CloudFront AWS services, unveiling their infrastructure, the pervasiveness of web services they host, and their traffic allocation policies as seen from our vantage points. Most importantly, we observe their evolution over a two-year-long period. The solution provided in this paper can be of interest for the following: 1) developers aiming at building measurement tools for cloud infrastructure providers; 2) developers interested in failure and anomaly detection systems; and 3) third-party service-level agreement certificators who can design systems to independently monitor performance. Finally, we believe that the results about AWS presented in this paper are interesting as they are among the first to unveil properties of AWS as seen from the operator point of view.

Network visibility is a critical part of traffic engineering, network management, and security. The most popular current solutions - Deep Packet Inspection (DPI) and statistical classification, deeply rely on the availability of a training set. Besides the cumbersome need to regularly update the signatures, their visibility is limited to classes the classifier has been trained for. Unsupervised algorithms have been envisioned as a viable alternative to automatically identify classes of traffic. However, the accuracy achieved so far does not allow to use them for traffic classification in practical scenario. To address the above issues, we propose SeLeCT, a Self-Learning Classifier for Internet Traffic. It uses unsupervised algorithms along with an adaptive seeding approach to automatically let classes of traffic emerge, being identified and labeled. Unlike traditional classifiers, it requires neither a-priori knowledge of signatures nor a training set to extract the signatures. Instead, SeLeCT automatically groups flows into pure (or homogeneous) clusters using simple statistical features. SeLeCT simplifies label assignment (which is still based on some manual intervention) so that proper class labels can be easily discovered. Furthermore, SeLeCT uses an iterative seeding approach to boost its ability to cope with new protocols and applications. We evaluate the performance of SeLeCT using traffic traces collected in different years from various ISPs located in 3 different continents. Our experiments show that SeLeCT achieves excellent precision and recall, with overall accuracy close to 98%. Unlike state-of-art classifiers, the biggest advantage of SeLeCT is its ability to discover new protocols and applications in an almost automated fashion.

Twitter has attracted millions of users that generate a humongous flow of information at constant pace. The research community has thus started proposing tools to extract meaningful information from tweets. In this paper, we take a different angle from the mainstream of previous works: we explicitly target the analysis of the timeline of tweets from "single users". We define a framework - named TUCAN - to compare information offered by the target users over time, and to pinpoint recurrent topics or topics of interest. First, tweets belonging to the same time window are aggregated into "bird songs". Several filtering procedures can be selected to remove stop-words and reduce noise. Then, each pair of bird songs is compared using a similarity score to automatically highlight the most common terms, thus highlighting recurrent or persistent topics. TUCAN can be naturally applied to compare bird song pairs generated from timelines of different users.

Content caching is a fundamental building block of the Internet. Caches are widely deployed at network edges to improve performance for end-users, and to reduce load on web servers and the backbone network. Considering mobile 3G/4G networks, however, the bottleneck is at the access link, where bandwidth is shared among all mobile terminals. As such, per-user capacity cannot grow to cope with the traffic demand. Unfortunately, caching policies would not reduce the load on the wireless link which would have to carry multiple copies of the same object that is being downloaded by multiple mobile terminals sharing the same access link.

Personal cloud storage services are data-intensive applications already producing a significant share of Internet traffic. Several solutions offered by different companies attract more and more people. However, little is known about each service capabilities, architecture and -- most of all -- performance implications of design choices. This paper presents a methodology to study cloud storage services. We apply our methodology to compare 5 popular offers, revealing different system architectures and capabilities. The implications on performance of different designs are assessed executing a series of benchmarks. Our results show no clear winner, with all services suffering from some limitations or having potential for improvement. In some scenarios, the upload of the same file set can take seven times more, wasting twice as much capacity. Our methodology and results are useful thus as both benchmark and guideline for system design.

This paper presents a characterization of Amazon's Web Services (AWS), the most prominent cloud provider that offers computing, storage, and content delivery platforms. Leveraging passive measurements, we explore the EC2, S3 and CloudFront AWS services to unveil their infrastructure, the pervasiveness of content they host, and their traffic allocation policies. Measurements reveal that most of the content residing on EC2 and S3 is served by one Amazon datacenter, located in Virginia, which appears to be the worst performing one for Italian users. This causes traffic to take long and expensive paths in the network. Since no automatic migration and load-balancing policies are offered by AWS among different locations, content is exposed to the risks of outages. The CloudFront CDN, on the contrary, shows much better performance thanks to the effective cache selection policy that serves 98% of the traffic from the nearest available cache. CloudFront exhibits also dynamic load-balancing policies, in contrast to the static allocation of instances on EC2 and S3. Information presented in this paper will be useful for developers aiming at entrusting AWS to deploy their contents, and for researchers willing to improve cloud design.

Network visibility is a critical part of traffic engineering, network management, and security. Recently, unsupervised algorithms have been envisioned as a viable alternative to automatically identify classes of traffic. However, the accuracy achieved so far does not allow to use them for traffic classification in practical scenario. In this paper, we propose SeLeCT, a Self-Learning Classifier for Internet traffic. It uses unsupervised algorithms along with an adaptive learning approach to automatically let classes of traffic emerge, being identified and (easily) labeled. SeLeCT automatically groups flows into pure (or homogeneous) clusters using alternating simple clustering and filtering phases to remove outliers. SeLeCT uses an adaptive learning approach to boost its ability to spot new protocols and applications. Finally, SeLeCT also simplifies label assignment (which is still based on some manual intervention) so that proper class labels can be easily discovered. We evaluate the performance of SeLeCT using traffic traces collected in different years from various ISPs located in 3 different continents. Our experiments show that SeLeCT achieves overall accuracy close to 98%. Unlike state-of-art classifiers, the biggest advantage of SeLeCT is its ability to help discovering new protocols and applications in an almost automated fashion.

Points of Presence (PoP), large aggregation nodes of a telecommunication network in which users lines are interconnected to the ISP backbone network, are relevant elements of the ISP network infrastructure. Motivated by the today interest of both ISPs and researchers to more energy efficient Internet, we investigate the power consumption of PoPs of FASTWEB, a national-wide ISP in Italy. Energy profiling spans a period of one year, and includes both ADSL and FTTH access technologies. This extensive and unique dataset allows us to shed light on energy consumption of ISP networks, which we profile against other measurements, such as external temperature and PoP handled traffic. Results show that energy consumption is independent on the traffic, while it is strongly correlated with both daily and annual variability of temperature, due to air conditioning energy cost. Starting from these results, we investigate some possible strategies to reduce ISP electricity bill. We consider the adoption of energy proportional architectures which are currently being investigated by both manufacturers and researchers. Moreover, we evaluate the possible energy savings using real traffic data and we obtain that simple PoPs energy saving models based on two-three energy operating configuration can achieve results comparable to fully energy proportional model.

A careful perusal of the Internet evolution reveals two major trends - explosion of cloud-based services and video streaming applications. In both of the above cases, the owner (e.g., CNN, YouTube, or Zynga) of the content and the organization serving it (e.g., Akamai, Limelight, or Amazon EC2) are decoupled, thus making it harder to understand the association between the content, owner, and the host where the content resides. This has created a tangled world wide web that is very hard to unwind, impairing ISPs' and network administrators' capabilities to control the traffic flowing in their networks.

Personal cloud storage services are gaining popularity. With a rush of providers to enter the market and an increasing offer of cheap storage space, it is to be expected that cloud storage will soon generate a high amount of Internet traffic. Very little is known about the architecture and the performance of such systems, and the workload they have to face. This understanding is essential for designing efficient cloud storage systems and predicting their impact on the network.

Traffic classification is still today a challenging problem given the ever evolving nature of the Internet in which new protocols and applications arise at a constant pace. In the past, so called behavioral approaches have been successfully proposed as valid alternatives to traditional DPI based tools to properly classify traffic into few and coarse classes. In this paper we push forward the adoption of behavioral classifiers by engineering a Hierarchical classifier that allows proper classification of traffic into more than twenty fine grained classes. Thorough engineering has been followed which considers both proper feature selection and testing seven different classification algorithms. Results obtained over actual and large data sets show that the proposed Hierarchical classifier outperforms off-the-shelf non hierarchical classification algorithms by exhibiting average accuracy higher than 90%, with precision and recall that are higher than 95% for most popular classes of traffic.

P2P-TV systems performance are driven by the overlay topology that peers form. Several proposals have been made in the past to optimize it, yet little experimental studies have corroborated results. The aim of this work is to provide a comprehensive experimental comparison of different strategies for the construction and maintenance of the overlay topology in P2P-TV systems. To this goal, we have implemented different fully-distributed strategies in a P2P-TV application, called Peer-Streamer, that we use to run extensive experimental campaigns in a completely controlled set-up which involves thousands of peers, spanning very different networking scenarios. Results show that the topological properties of the overlay have a deep impact on both user quality of experience and network load. Strategies based solely on random peer selection are greatly outperformed by smart, yet simple strategies that can be implemented with negligible overhead. Even with different and complex scenarios, the neighborhood filtering strategy we devised as most performing guarantees to deliver almost all chunks to all peers with a play-out delay as low as only 6s even with system loads close to 1.0. Results are confirmed by running experiments on PlanetLab. PeerStreamer is open-source to make results reproducible and allow further research by the community.

In the recent years, the research community has increased its focus on network monitoring which is seen as a key tool to understand the Internet and the Internet users. Several studies have presented a deep characterization of a particular application, or a particular network, considering the point of view of either the ISP, or the Internet user. In this paper, we take a different perspective. We focus on three European countries where we have been collecting traffic for more than a year and a half through 5 vantage points with different access technologies. This humongous amount of information allows us not only to provide precise, multiple, and quantitative measurements of "What the user do with the Internet" in each country but also to identify common/uncommon patterns and habits across different countries and nations. Considering different time scales, we start presenting the trend of application popularity; then we focus our attention to a one-month long period, and further drill into a typical daily characterization of users activity. Results depict an evolving scenario due to the consolidation of new services as Video Streaming and File Hosting and to the adoption of new P2P technologies. Despite the heterogeneity of the users, some common tendencies emerge that can be leveraged by the ISPs to improve their service.

According to several studies, the power consumption of the Internet accounts for up to 10% of the worldwide energy consumption and is constantly increasing. The global consciousness on this problem has also grown, and several initiatives are being put into place to reduce the power consumption of the ICT sector in general. In this paper, we face the problem of minimizing power consumption for Internet service provider (ISP) networks. In particular, we propose and assess strategies to concentrate network traffic on a minimal subset of network resources. Given a telecommunication infrastructure, our aim is to turn off network nodes and links while still guaranteeing full connectivity and maximum link utilization constraints. We first derive a simple and complete formulation, which results into an NP-hard problem that can be solved only for trivial cases. We then derive more complex formulations that can scale up to middle-sized networks. Finally, we provide efficient heuristics that can be used for large networks. We test the effectiveness of our algorithms on both real and synthetic topologies, considering the daily fluctuations of Internet traffic and different classes of users. Results show that the power savings can be significant, e.g., larger than 35%.

We conduct a detailed simulation study to examine how localizing P2P traffic within network boundaries impacts the profitability of an ISP. A distinguishing aspect of our work is the focus on Internet-wide implications, i.e., how adoption of localization within an ISP affects both itself and other ISPs. Our simulations are based on detailed models that estimate inter-autonomous-system (AS) P2P traffic and inter-AS routing, localization models that predict the extent to which P2P traffic is reduced, and pricing models that predict the impact of changes in traffic on the profit of an ISP. We evaluate our models by using a large-scale crawl of BitTorrent containing over 138 million users sharing 2.75 million files. Our results show that the benefits of localization must not be taken for granted. Some of our key findings include: 1) residential ISPs can actually lose money when localization is employed, and some of them will not see increased profitability until other ISPs employ localization; 2) the reduction in costs due to localization will be limited for small ISPs and tends to grow only logarithmically with client population; and 3) some ISPs can better increase profitability through alternate strategies to localization by taking advantage of the business relationships they have with other ISPs.

Asymmetric Digital Subscriber Line - ADSL - technology is the preferred high-speed access to the Internet, offering up to 24Mb/s on downlink channels. Exploiting the copper media already deployed by Telecom Operators, physical link quality often limits the maximum ADSL data rate, so that the actual bitrate of a line is often unpredictable. In this work, we analyze a large set of end-users' ADSL links, and try to correlate the effective bitrate with the physical measurements exposed by network devices. By exploiting a Neural Network (NN) predictor, we acquire knowledge about the behavior of ADSL lines from the huge amount of measured data. We show that NNs are good tools to automatically predict end-users' ADSL available bandwidth. However some ingenuity is required to guide the learning phase and to avoid misbehaving lines to fool the NN prediction.

In this paper, we conduct a detailed study of the YouTube CDN with a view to understanding the mechanisms and policies used to determine which data centers users download video from. Our analysis is conducted using week-long datasets simultaneously collected from the edge of five networks - two university campuses and three ISP networks - located in three different countries. We employ state-of-the-art delay-based geolocation techniques to find the geographical location of YouTube servers. A unique aspect of our work is that we perform our analysis on groups of related YouTube flows. This enables us to infer key aspects of the system design that would be difficult to glean by considering individual flows in isolation. Our results reveal that while the RTT between users and data centers plays a role in the video server selection process, a variety of other factors may influence this selection including load-balancing, diurnal effects, variations across DNS servers within a network, limited availability of rarely accessed video, and the need to alleviate hot-spots that may arise due to popular video content.

In this paper we present a complete measurement study that compares YouTube traffic generated by mobile devices (smart-phones,tablets) with traffic generated by common PCs (desktops, notebooks, netbooks). We investigate the users' behavior and correlate it with the system performance. Our measurements are performed using unique data sets which are collected from vantage points in nation-wide ISPs and University campuses from two countries in Europe and the U.S.

In this paper we present an extensive analysis of traffic generated by SopCast users and collected from operative networks of three national ISPs in Europe. After more than a year of continuous monitoring, we present results about the popularity of SopCast which is the largely preferred application in the studied networks. We focus on analysis of (i) application and bandwidth usage at different time scales, (ii) peer lifetime, arrival and departure processes, (iii) peer localization in the world. Results provide useful insights into users' behavior, including their attitude towards P2P-TV application usage and the consequent generated load on the network, that is quite variable based on the access technology and geographical location. Our findings are interesting to Researchers interested in the investigation of users' attitude towards P2P-TV services, to foresee new trends in the future usage of the Internet, and to augment the design of their application.

In this paper we consider mesh based P2P streaming systems focusing on the problem of regulating peer upload rate to match the system demand while not overloading each peer upload link capacity. We propose Hose Rate Control (HRC), a novel scheme to control the speed at which peers offer chunks to other peers, ultimately controlling peer uplink capacity utilization. This is of critical importance for heterogeneous scenarios like the one faced in the Internet, where peer upload capacity is unknown and varies widely. HRC nicely adapts to the actual peer available upload bandwidth and system demand, so that users' Quality of Experience is greatly enhanced. Both simulations and actual experiments involving up to 1000 peers are presented to assess performance in real scenarios. Results show that HRC consistently outperforms the Quality of Experience achieved by non-adaptive schemes.

Several successful commercial P2P-TV applications are already available. Unfortunately, some algorithms and protocols they adopt are unknown, since many follow a closed and proprietary design. This calls for tools and methodologies that allow the investigation of the application behavior. In this paper, we present a novel approach to analyze the graph properties and the traffic generated by P2P-TV applications run by customers in operative networks. The proposed methodology allows us to distinguish and investigate three different graphs: (i) the social networks that link users based on their interest, (ii) the overlay networks created by peers that are watching the same channel, and (iii) the distribution networks that involve the subset of peers that are contributing to the video distribution. We apply this methodology to the traffic collected for more than one year from three national ISPs in Europe, where SopCast is the largely preferred application. Considering users' behavior, we uncover the attitude to use the P2P-TV application mainly to follow live sport events. P2P-TV systems have then to deal with both flash crowd and sudden peer departures that happen at the beginning and end of an event. Furthermore, zapping among channels offering the same event is also relevant. SopCast deals with this by implementing a very robust and greedy overlay topology discovery process in which more than 170 peers are contacted every 60 s. Considering video distribution, we provide evidence that SopCast implements algorithms that restrict traffic within Autonomous System boundaries. Still, high bandwidth peers must be present to supply the necessary upload capacity to sustain the video service.

In this paper, we investigate the impact of peer bandwidth heterogeneity on the performance of a mesh-based P2P system for live streaming. We show that bandwidth heterogeneity constitutes an important resource for P2P live streaming systems. Indeed, by effectively exploiting it, the overall performance of the system is significantly improved. This requires the adoption of smart schemes for both the overlay topology construction and chunk scheduling mechanisms that discriminate among peers based on their bandwidth.

The end-to-end available bandwidth of an Internet path is a desirable information that can be exploited to optimize system performance. Several tools have been proposed in the past to estimate it. However, existing measurement techniques were not designed for large-scale deployments. In this paper we show that current tools do not properly work where multiple probing processes share a portion of a path. We provide experimental evidence to quantify the impact of mutual interference between measurements. We further analyze the characteristics of popular tools, quantifying (i) the impact of mutual interference, (ii) the total overhead imposed to the network and (iii) the intrusiveness of the measurement process in a large-scale scenario. Our goal is to effectively quantify the impact of concurrent measurements on current estimation techniques and to offer some simple guidelines for dimensioning a large-scale measurement system.

We consider large-scale mesh-based P2P systems for the distribution of real-time video content. Our goal is to study the impact that different design choices adopted while building the overlay topology may have on the system performance. In particular, we show that the adoption of different strategies leads to overlay topologies with different macroscopic properties. Representing the possible overlay topologies with different families of random graphs, we develop simple, yet accurate, fluid models that capture the dominant dynamics of the chunk distribution process over several families of random graphs. Our fluid models allow us to compare the performance of different strategies providing a guidance for the design of new and more efficient systems. In particular, we show that system performance can be significantly improved when possibly available information about peers location and/or peer access bandwidth is carefully exploited in the overlay topology formation process.

Peer-to-Peer streaming systems (or P2P-TV) have been studied in the literature for some time, and they are becoming popular among users as well. P2P- TV systems target the real time delivery of a video stream, therefore posing different challenges compared to more traditional P2P applications like the better known file sharing P2P application. In this paper, we focus on mesh based systems in which the peers form a generic overlay topology upon which peers exchange small "chunks" of video. In particular, we study the signaling mechanisms that must be in place to trade chunks and to match the demand from other peers in a quick and efficient way, by automatically adapting a peer service rate to its upload capacity. The goal is to maximize peer upload capacity utilization, while avoiding forming long transmission queue, therefore minimizing the chunk delivery time, a crucial parameter for P2P-TV systems. Our results show that the proposed solution achieves several desirable goals: i) it limits the overhead due to signaling messages, ii) it achieves a fair resource utilization, making peers contribute proportionally to their bandwidth, iii) it improves system performance, reducing loss probability and chunk delivery delay with respect to mechanisms with non adaptive number of contacted peers.

Abstract—Peer-to-Peer IP Television (P2PTV) applications represent one of the fastest growing application classes on the Internet, both in terms of their popularity and in terms of the amount of traffic they generate. While network operators require monitoring tools that can effectively analyze the traffic produced by these systems, few techniques have been tested on these mostly closed-source, proprietary applications. In this paper we examine the properties of three traffic classifiers applied to the problem of identifying P2PTV traffic. We report on extensive experiments conducted on traffic traces with reliable ground truth information, highlighting the benefits and shortcomings of each approach. The results show that not only their performance in terms of accuracy can vary significantly, but also that their usability features suggest different effective aspects that can be integrated. I.

In this paper, we extend the concept of Stochastic Packet Inspection (SPI) to support TCP traffic classification. SPI is a method based on the statistical fingerprint of the application-layer headers: by characterizing the frequencies of observed symbols, SPI can identify application protocol formats by automatically recognizing group of bits that take e.g., constant values, or random values, or are part of a counter. To correctly characterize symbol frequencies, SPI needs volumes of traffic to obtain statistically significant signatures. Earlier proposed for UDP traffic, SPI has to be modified to cope with the connection oriented service offered by TCP, in which application-layer headers are only found at the beginning of a TCP connection. In this paper, we extend SPI to support TCP traffic, and analyze its performance on real network data. The key idea is to move the classification target from single flows to endpoints, which aggregates all traffic sent/received by the same IP address and TCP port pair. The first few packets of flows sent from (or destined to) the same endpoint are then aggregated to yield a single SPI signature. Results show that SPI is able to achieve remarkably good results, with an average true positive rate of about 98%.

P2P-TV systems have become part of the Internet landscape (See for instance http://www.pplive.com, http://www.soapcast.com, http://www.tvants.com, and many others). The architecture of these (normally proprietary) applications is generally receiver-driven, in that receivers actively search for suitable peers to download from, trying to maximize their performance. This results in very aggressive applications that generate huge and non optimized traffic loads. The demo summarized in this short paper shows the impact of various P2P streaming options and the efficiency of the Napa-Wine approach (compared to more traditional approaches) by running real streaming clients in realistic conditions. To make this comparison possible, the software developed in Napa-Wine is highly modular and configurable, allowing the user to test different topology management and chunk trading techniques developed within the Napa-Wine project, as well as to configure it to mimic other chunk/peer selection strategies known from literature.

This paper presents a systematic performance analysis of pull P2P video streaming systems for live applications, providing guidelines for the design of the overlay topology and the chunk scheduling algorithm. The contribution of the paper is threefold: (1) we propose a realistic simulative model of the system that represents the effects of access bandwidth heterogeneity, latencies, peculiar characteristics of the video, while still guaranteeing good scalability properties; (2) we propose a new latency/bandwidth-aware overlay topology design strategy that improves application layer performance while reducing the underlying transport network stress; (3) we investigate the impact of chunk scheduling algorithms that explicitly exploit properties of encoded video. Results show that our proposal jointly improves the actual Quality of Experience of users and reduces the cost the transport network has to support.

Early P2P-TV systems have already attracted millions of users, and many new commercial solutions are entering this market. Little information is however available about how these systems work, due to their closed and proprietary design. In this paper, we present large scale experiments to compare three of the most successful P2P-TV systems, namely PPLive, SopCast and TVAnts. Our goal is to assess what level of "network awareness" has been embedded in the applications. We first define a general framework to quantify which network layer parameters leverage application choices, i.e., what parameters mainly drive the peer selection and data exchange. We then apply the methodology to a large dataset, collected during a number of experiments where we deployed about 40 peers in several European countries. From analysis of the dataset, we observe that TVAnts and PPLive exhibit a mild preference to exchange data among peers in the same autonomous system the peer belongs to, while this clustering effect is less intense in SopCast. However, no preference versus country, subnet or hop count is shown. Therefore, we believe that next-generation P2P live streaming applications definitively need to improve the level of network-awareness, so to better localize the traffic in the network and thus increase their network-friendliness as well.

This paper proposes KISS, a novel Internet classification engine. Motivated by the expected raise of UDP traffic, which stems from the momentum of Peer-to-Peer (P2P) streaming applications, we propose a novel classification framework that leverages on statistical characterization of payload. Statistical signatures are derived by the means of a Chi-Square (χ2)-like test, which extracts the protocol “format,” but ignores the protocol “semantic” and “synchronization” rules. The signatures feed a decision process based either on the geometric distance among samples, or on Support Vector Machines. KISS is very accurate, and its signatures are intrinsically robust to packet sampling, reordering, and flow asymmetry, so that it can be used on almost any network. KISS is tested in different scenarios, considering traditional client-server protocols, VoIP, and both traditional and new P2P Internet applications. Results are astonishing. The average True Positive percentage is 99.6%, with the worst case equal to 98.1,% while results are almost perfect when dealing with new P2P streaming applications.

In this paper, Internet data collected via passive measurement are analyzed to obtain localization information on nodes by clustering (i.e., grouping together) nodes that exhibit similar network path properties. Since traditional clustering algorithms fail to correctly identify clusters of homogeneous nodes, we propose a novel framework, named "NetCluster", suited to analyze Internet measurement datasets. We show that the proposed framework correctly analyzes synthetically generated traces. Finally, we apply it to real traces collected at the access link of our campus LAN and discuss the network characteristics as seen at the vantage point.

According to several studies, the power consumption of the Internet accounts for up to 10% of the worldwide energy consumption, and several initiatives are being put into place to reduce the power consumption of the ICT sector in general. To this goal, we propose a novel approach to switch off network nodes and links while still guaranteeing full connectivity and maximum link utilization. After showing that the problem falls in the class of capacitated multi-commodity flow problems, and therefore it is NP-complete, we propose some heuristic algorithms to solve it. Simulation results in a realistic scenario show that it is possible to reduce the number of links and nodes currently used by up to 30% and 50% respectively during off-peak hours, while offering the same service quality.

Skype is one of the most successful VoIP application in the current Internet spectrum. One of the most peculiar characteristics of Skype is that it relies on a P2P infrastructure for the exchange of signaling information amongst active peers. During August 2007, an unexpected outage hit the Skype overlay, yielding to a service blackout that lasted for more than two days: this paper aims at throwing light to this event. Leveraging on the use of an accurate Skype classification engine, we carry on an experimental study of Skype signaling during the outage. In particular, we focus on the signaling traffic before, during and after the outage, in the attempt to quantify interesting properties of the event. While it is very difficult to gather clear insights concerning the root causes of the breakdown itself, the collected measurement allow nevertheless to quantify several interesting aspects of the outage: for instance, measurements show that the outage caused, on average, a 3-fold increase of signaling traffic and a 10-fold increase of number of contacted peers, topping to more than 11 million connections for the most active node in our network - which immediately gives the feeling of the extent of the phenomenon.

In this paper we define a simple experimental setup to analyze the behavior of commercial P2P-TV applications under adverse network conditions. Our goal is to reveal the ability of different P2P-TV applications to adapt to dynamically changing conditions, such as delay, loss and available capacity, e.g., checking whether such systems implement some form of congestion control. We apply our methodology to four popular commercial P2P-TV applications: PPLive, SOPCast, TVants and TVUPlayer. Our results show that all the considered applications are in general capable to cope with packet losses and to react to congestion arising in the network core. Indeed, all applications keep trying to download data by avoiding bad paths and carefully selecting good peers. However, when the bottleneck affects all peers, e.g., it is at the access link, their behavior results rather aggressive, and potentially harmful for both other applications and the network.

Early P2P-TV systems have already attracted millions of users, and many new commercial solutions are entering this market. Little information is however available about how these systems work. In this paper we present large scale sets of experiments to compare three of the most successful P2P-TV systems, namely PPLive, SopCast and TVAnts. Our goal is to assess what level of "network awareness" has been embedded in the applications, i.e., what parameters mainly drive the peer selection and data exchange. By using a general framework that can be extended to other systems and metrics, we show that all applications largely base their choices on the peer bandwidth, i.e., they prefer high-bandwidth users, which is rather intuitive. Moreover, TVAnts and PPLive exhibits also a preference to exchange data among peers in the same autonomous system the peer belongs to. However, no evidence about preference versus peers in the same subnet or that are closer to the considered peer emerges. We believe that next-generation P2P live streaming applications definitively need to improve the level of network-awareness, so to better localize the traffic in the network and thus increase their network-friendliness as well.

In this paper, we propose a simple and fully distributed mechanism for constructing and maintaining the overlay topology in mesh-based P2P-TV systems. Our algorithm optimizes the topology to better exploit large bandwidth peers, so that they are automatically moved close to the source. This improves the chunk delivery delay so that all peers benefit, not just the high bandwidth ones. A key property of the proposed scheme is its ability to indirectly estimate the upload bandwidth of peers without explicitly knowing or measuring it. Simulation results show that our scheme significantly outperforms overlays with homogeneous properties, achieving up to 50% performance improvement. Moreover, the algorithm is robust to both parameter setting and changing conditions, e.g., peer churning.

Skype is beyond any doubt the VoIP application in the current Internet application spectrum. Its amazing success has drawn the attention of telecom operators and the research community, both interested in knowing its internal mechanisms, characterizing its traffic, understanding its users' behavior. In this paper, we investigate the characteristics of traffic streams generated by voice and video communications, and the signaling traffic generated by Skype. Our approach is twofold, as we make use of both active and passive measurement techniques to gather a deep understanding on the traffic Skype generates. From extensive testbed experiments, we devise a source model which takes into account: i) the service type, i.e., SkypeOut calls or calls between two Skype clients, ii) the selected source Codec, iii) the adopted transport layer protocol, and iv) network conditions. Leveraging on the use of an accurate Skype classification engine that we recently proposed, we study and characterize Skype traffic based on extensive passive measurements collected from our campus LAN.

In this paper, we consider two popular peer-to-peer TV (P2P-TV) systems: PPLive, one of the today most widely used P2P-TV systems, and Joost, a promising new generation application of which no previous measurement study has been considered. Besides the traditional measurements like the amount of generated traffic for signaling and data transmission, the novel contribution of the paper consists in investigating the content distribution mechanisms. In particular, we evaluate the characteristics of both data distribution and signaling process for the overlay network discovery and maintenance. By considering two or more clients in the same sub-network, we observe the capability of the system to exploit the locality of peers. We also explore how the system adapts to different network conditions. The methodology we develop allows also to identify periodic behavior of the application, highlighting bursts of both data and signaling traffic.

Skype is beyond any doubt the most popular VoIP application in the current Internet application spectrum. Its amazing success drawn the attention of telecom operators and the research community, both interested in knowing Skype's internal mechanisms, characterizing traffic and understanding users' behavior. We dissect the following fundamental components: data traffic generated by voice and video communication, and signaling traffic generated by Skype. We use both active and passive measurement techniques to gather a deep understanding on the traffic Skype generates. From extensive testbed experiments, we devise a source model which takes into account: (i) the service type, i.e., voice or video calls (ii) the selected source Codec, (iii) the adopted transport-layer protocol, and (iv) network conditions. Furthermore, leveraging on the use of an accurate Skype classification engine that we recently proposed, we study and characterize Skype traffic based on extensive passive measurements collected from our campus LAN.

P2P-TV systems distribute live streaming contents by organizing the information flow in small chunks that are exchanged among peers. Different strategies can be implemented at the peers to select the chunk to distribute and the destination neighboring peer. Recent work showed that a good strategy consists in selecting the latest received chunk and a random neighboring peer (latest useful chunk, random peer). In this paper, leveraging on the idea that it is convenient to favor those peers that can contribute the most to the chunk distribution, we propose to select the destination peer with a probability proportional to the peer upload bandwidth. We show that the proposed scheme has a limited sensitivity to cheating peers that maliciously declare higher bandwidth than they actually have. Considering the overlay topology, we evaluate both systems in which nodes have fixed degree and systems whose overlay setup takes into account the actual peer bandwidth by assigning more neighbors to peer with higher bandwidth. We evaluate the performance in terms of delay percentiles and loss probability and evaluate the achieved improvements. Simulation results considering scenarios with up to 10,000 peers shows that the proposed schemes significantly outperform the traditional ones, so that the chunk distribution delay drops to less than 2 s from about 12 s.

VoIP has widely been addressed as the technology that will change the Telecommunication model opening the path for convergence. Still today this revolution is far from being complete, since the majority of telephone calls are originated by circuit-oriented networks. In this paper for the first time to the best of our knowledge, we present a large dataset of measurements collected from the FastWeb backbone, which is one of the first worldwide Telecom operator to offer VoIP and high-speed data access to the end-user. Traffic characterization will focus on several layers, focusing on both end-user and ISP perspective. In particular, we highlight that, among loss, delay and jitter, only the first index may affect the VoIP call quality. Results show that the technology is mature to make the final step, allowing the integration of data and real-time services over the Internet.

In this paper, we present measurement results collected from real traces on the network FastWeb, an ISP provider that is the main broadband telecommunication company in Italy. The network relies on a fully IP architecture and delivers to the user services such as data, VoIP and IP television over a single broadband connection. Our measurements, that are based on a passive measurement technique, focus on IP Television (IPTV) multicast, that consists of 83 digital TV channels encoded using different MPEG-2 encoders. The results show that, depending on the encoder and based on the bitrate, flows can be classified as being: CBR, 2-VBR (i.e., two typical bitrate values) and VBR. Measurement of the packet loss, jitter and inter-packet gap show that, independently from the class, packet generation process of the flows can have various degrees of burstiness. Despite the packet level burstiness, average jitter is limited to few milliseconds and no packet loss was ever observed, showing that the quality of IPTV offered by FastWeb is excellent.

Skype is a very popular VoIP software which has recently attracted the attention of the research community and network operators. Following a closed source and proprietary design, Skype protocols and algorithms are unknown. Moreover, strong encryption mechanisms are adopted by Skype, making it very difficult to even glimpse its presence from a traffic aggregate. In this paper, we propose a framework based on two complementary techniques to reveal Skypetraffic in real time. The first approach, based on Pearson'sChi-Square test and agnostic to VoIP-related trafficcharacteristics, is used to detect Skype's fingerprint from the packet framing structure, exploiting the randomness introduced at the bit level by the encryption process. Conversely, the second approach is based on a stochastic characterization of Skype traffic in terms of packet arrival rate and packet length, which are used as features of a decision process based on Naive Bayesian Classifiers.In order to assess the effectiveness of the above techniques, we develop an off-line cross-checking heuristic based on deep-packet inspection and flow correlation, which is interesting per se. This heuristic allows us to quantify the amount of false negatives and false positives gathered by means of the two proposed approaches: results obtained from measurements in different networks show that the technique is very effective in identifying Skype traffic. While both Bayesian classifier and packet inspection techniques are commonly used, the idea of leveraging on randomness to reveal traffic is novel. We adopt this to identify Skype traffic, but the same methodology can be applied to other classification problems as well.

Software routers are becoming an important alternative to proprietary and expensive network devices, because they exploit the economy of scale of the PC market and open-source software. When considering maximum performance in terms of throughput, PC-based routers suffer from limitations stemming from the single PC architecture, e.g., limited bus bandwidth, and high memory access latency. To overcome these limitations, in this paper we present a multistage architecture that combines a layer-2 load-balancer front-end and a layer-3 routing back-end, interconnected by standard Ethernet switches. Both the front-end and the back-end are implemented using standard PCs and open- source software. After describing the architecture, evaluation is performed on a lab test-bed, to show its scalability. While the proposed solution allows to increase performance of PC- based routers, it also allows to distribute packet manipulation functionalities, and to automatically recover from component failures.

In this paper we focus on passive measurements of TCP traffic, main component of nowadays traffic. We propose a heuristic technique for the classification of the anomalies that may occur during the lifetime of a TCP flow, such as out-of-sequence and duplicate segments. Since TCP is a closed-loop protocol that infers network conditions by means of losses and reacts accordingly, the possibility of carefully distinguishing the causes of anomalies in TCP traffic is very appealing, since it may be instrumental to the deep understanding of TCP behavior in real environments and to protocol engineering as well. We apply the proposed heuristic to traffic traces collected at both networks edges and backbone links. By studying the statistical properties of TCP anomalies, we find that their aggregate exhibits Long Range Dependence phenomena, but that anomalies suffered by individual long-lived flows are on the contrary uncorrelated. Interestingly, no dependence to the actual link load is observed.

Traffic measurement represents an indispensable and valuable tool for the analysis of nowadays telecommunication networks. Moreover, it is desirable for traffic measurement and analysis to be both continuous and persistent, since only these joint requirements allow to track important changes on the traffic pattern. On the other hand, transmission links bandwidth keep improving, at a seemingly inexorable rate: therefore, the analysis of the traffic is becoming more complex than ever. This paper focuses on the description and the benchmarking of a network traffic analyzer, called Tstat, able to process real-time traffic further providing i) several advanced measurement indexes of transport layer protocols and ii) ever-lasting monitoring capabilities. Particularly, our aim is to assess what kind of links, and under which load, can be continuously and persistently monitored without compromising the complexity of the traffic analysis that has to be performed.

We focus on the identification and definition of "Web user-sessions", an aggregation of several TCP connections generated by the same source host on the basis of TCP connection opening time. The identification of a user session is non trivial; traditional approaches rely on threshold based mechanisms, which are very sensitive to the value assumed for the threshold and may be difficult to correctly set. By applying clustering techniques, we define a novel methodology to identify Web user-sessions without requiring an a priori definition of threshold values. We analyze the characteristics of user sessions extracted from real traces, studying the statistical properties of the identified sessions. From the study it emerges that Web user-sessions tend to be Poisson, but correlation may arise during periods of network/hosts anomalous functioning.

In this paper we consider packet networks loaded by admissible traffic patterns, i.e. by traffic patterns that, if optimally routed, do not overload network resources. In these conditions, we study the combined behavior of distributed dynamic routing and scheduling algorithms based upon link state information, with no knowledge of the average traffic pattern, and we prove that simple schemes can achieve the same network throughput as optimal centralized routing and scheduling algorithms with complete information on the traffic pattern. Our study is based on a flow-level abstract model of the network, and considers elastic traffic, i.e., we assume that flows can adapt their transmission rates to network conditions. As a result, our model captures some of the main features of Internet traffic and of quality-of-service routing approaches being currently proposed for IP networks. We show that efficient dynamic routing and scheduling algorithms can be implemented in a distributed way, and we prove that maximum throughput is achieved also in case of temporary mismatches between the actual link metrics and those used by the routing algorithm. This is a particularly relevant aspect, since any distributed implementation of a routing algorithm requires a periodic exchange of link state information among nodes, and this implies delays, and thus time periods in which the actual link state is not known.

In this correspondence, we discuss the stability of scheduling algorithms for input-queueing (IQ) and combined input/output queueing (CIOQ) packet switches. First, we show that a wide class of IQ schedulers operating on multiple traffic classes can achieve 100% throughput. Then, we address the problem of the maximum throughput achievable in a network of interconnected IQ switches and CIOQ switches loaded by multiclass traffic, and we devise some simple scheduling policies that guarantee 100% throughput. Both the Lyapunov function methodology and the fluid modeling approach are used to obtain our results.

TCP Smart Framing, or TCP-SF for short, enables the Fast Retransmit/Recovery algorithms even when the congestion window is small. Without modifying the TCP congestion control based on the additive-increase/multiplicative-decrease paradigm, TCP-SF adopts a novel segmentation algorithm: while Classic TCP always tries to send full-sized segments, a TCP-SF source adopts a more flexible segmentation algorithm to try and always have a number of in-flight segments larger than 3 so as to enable Fast Recovery. We motivate this choice by real traffic measurements, which indicate that today's traffic is populated by short-lived flows, whose only means to recover from a packet loss is by triggering a Retransmission Timeout. The key idea of TCP-SF can be implemented on top of any TCP flavor, from Tahoe to SACK, and requires modifications to the server TCP stack only, and can be easily coupled with recent TCP enhancements. The performance of the proposed TCP modification were studied by means of simulations, live measurements and an analytical model. In addition, the analytical model we have devised has a general scope, making it a valid tool for TCP performance evaluation in the small window region. Improvements are remarkable under several buffer management schemes, and maximized by byte-oriented schemes.

In this paper, we propose a MMPP (Markov modulated Poisson process) traffic model that accurately approximates the LRU (long range dependence) characteristics of Internet traffic traces. Using the notion of sessions and flows, the proposed MMPP model mimics the real hierarchical behavior of the packet generation process by Internet users. Thanks to its hierarchical structure, the proposed model is both simple and intuitive: it allows the generation of traffic with the desired characteristics by easily setting a few input parameters which have a clear physical meaning. Results prove that the queuing behavior of the traffic generated by the MMPP model is coherent with the one produced by the real traces collected at our institution edge router under different networking scenarios and loads. Due to its characteristics, the proposed MMPP traffic model can be used as a simple and manageable tool for IP network performance analysis, as well as for network planning and dimensioning.

We study the TCP flow arrival process, starting from the aggregated measurement at the TCP flow level taken from our campus network. In particular, we analyze the statistical properties of the TCP flow arrival process. We define the different traffic aggregates by splitting the original trace, such that i) each of them is constituted by all the TCP flows belonging to the same traffic relation, i.e., with the same source/destination IP addresses and ii) each traffic aggregate has, bytewise, the same amount of traffic. To induce a divisions of TCP-elephants and TCP-mice into different traffic aggregates, the used algorithm packs the largest traffic relations in the first traffic aggregates, so that subsequently generated aggregates are constituted by an increasing number of smaller traffic relations. The long range dependency (LRD) characteristics are presented, showing the possible causes of the LRD of TCP flow arrival process in i) the heavy tailed distribution of the number of flows in a traffic aggregate, and ii) the presence of TCP-elephants within them.

We present a study of Web user behavior when network performance decreases causing an increase of page transfer times. Real traffic measurements are analyzed to infer whether worsening network conditions translate into greater impatience by the user, which translates in early interruption of TCP connections. Several parameters are studied in order to gather their impact on the interruption probability on Web transfers: time of day, file size, throughput and time elapsed since the beginning of the download. From the results presented, we try to paint a picture of the complex interactions between user perception of the Web and network-level events.

In this article, we develop a general methodology, mainly based upon Lyapunov functions, to derive bounds on average delays, and on averages and variances of queue lengths in complex systems of queues. We apply this methodology to cell-based switches and routers, considering first output-queued (OQ) architectures, in order to provide a simple example of our methodology, and then both input-queued (IQ), and combined input/output queued (CIOQ) architectures. These latter switching architectures require a scheduling algorithm to select at each slot a subset of input-buffered cells that can be transferred toward output ports. Although the stability properties (i.e., the limit throughput) of IQ and CIOQ cell-based switches were already studied for several classes of scheduling algorithms, very few analytical results concerning cell delays or queue lengths are available in the technical literature. We concentrate on Maximum Weight Matching (MWM) and Maximal Size Matching (mSM) scheduling algorithms; while the former was proved to maximize throughput, the latter allows simpler implementation. The derived bounds are shown to be rather tight when compared to simulation results.

Tstat is a new tool for the collection and statistical analysis of TCP/IP traffic, able to infer TCP connection status from traces. Discussing its use, we present some of the performance figures that can be obtained and the insight that such figures can give on TCP/IP protocols and the Internet. While standard performance measure, such as flow dimensions, traffic distribution, etc., remain at the base of traffic evaluation, more sophisticated indices, like the out-of-order probability and gap dimension in TCP connections, obtained through data correlation between the incoming and outgoing traffic, give reliable estimates of the network performance also from the user perspective. Several of these indices are discussed on traffic measures performed for more than 2 months on the access link of our institution.

We define an analytical approach and a methodology to determine the set of service level agreements (SLA) that can be effectively supported by a DiffServ IP network. We consider the assured forwarding (AF) per hop behavior (PHB), and, based on the SLA probabilistic description, we derive a worst-case mathematical formulation for the overbooking probability, i.e., the probability that the traffic crossing any link of a source-destination path exceeds the link capacity. We then compare analytical results with simulation of different traffic models.

We present a simulation study of HTTP traffic crossing a DiffServ domain. We consider both the cases where the reserved bandwidth is not exceeded by the offered traffic (overprovisioning) and where the assured traffic competes with the classic best effort class (underprovisioning). The reported simulation shows that the DiffServ approach is able to protect the assured flows in the first case, while the performance benefits are tighter in the second case, in which fairness issues arise between long and short-lived flows.

This paper discusses a new approach to QoS routing, introducing the notion of algorithm resilience (i.e., its capability to adapt to network and load modifications) as the performance index of the algorithm itself, for a given network topology, load and traffic pattern. The new approach can be summarized as network graph reduction, i.e., a modification of the graph describing the network before the routing path is computed, in order to exclude from the path selection over-congested portions of the network. This solution leads to a class of two-step routing algorithms, where both steps are simple, hence allowing efficient implementation. Simulation experiments, run on randomly-generated topologies and traffic patterns, show that these routing algorithms outperform both the standard minimum hop algorithm and those QoS-based algorithms based on the same metrics but not using the notion of network graph reduction.

Many studies provide an extended investigation of the maximum throughput achievable in input-queueing (IQ) or combined-input-and-output-queueing (CIOQ) packet switches. Some scheduling policies, among which are maximum weight matching algorithms, were identified as optimal, in the sense that they were proved to achieve 100% throughput under any admissible single-class traffic pattern. Most of the results in the literature, however, consider just one switch in isolation, operating on packets belonging to a single traffic class. In this paper we first generalize known results, showing that a wide class of IQ schedulers operating on multiple traffic classes can achieve 100% throughput. In addition, we address the problem of the maximum throughput achievable in a network of interconnected IQ switches loaded by multiclass traffic, and we devise some simple scheduling policies that guarantee 100% throughput when switches are interconnected in a network. Both the Lyapunov function methodology and the fluid models approach are used to obtain our results.

Wavelength routed optical networks allow to design a logical topology, comprising lightpaths and routers, which is overlayed on the physical topology, comprising optical fibers and optical cross-connects, by solving a routing and wavelength assignment (RWA) problem. In this paper we extend the concept of lightpath to the one of super-lightpath, which uses a simple bit level time division multiplexing that can be directly implemented in the optical domain, to split the wavelength bandwidth among more than one traffic flow. This allows to design logical topologies with an increased number of logical links, thus reducing the average distance among nodes, i.e., the number of electro-optic and opto-electronic conversions, and the traffic congestion on logical links. At the same time, this reduces the number of wavelengths required to solve the RWA problem. Being the super-lightpath RWA problem computationally intractable, we propose two heuristics which show that the number of wavelengths required to overlay the same logical topology on the same physical topology is reduced by more that 65% using super-lightpaths.

This paper addresses the problem of evaluating routing algorithms via simulation in packet-switched networks when elastic traffic is involved. It highlights some deficiencies of classical approaches that fail to capture both the complex interactions of connections traversing multiple bottlenecks and common user behaviors. The paper describes an approach devised to overcome these limitations which is particularly suited for the evaluation of routing algorithms in presence of best-effort traffic. The simulation results presented offer a deeper insight into well-known routing algorithms. Through this analysis it is clear that quantitative and also qualitative behaviors of dynamic routing algorithms based on traffic measurements may be fairly different depending on the nature of the traffic loading the network, as well as depending on its interactions with the network parameters and behavior.

In this paper we propose an enhancement to the TCP protocol, called TCP Smart-Framing(TCP-SF), that enables the Fast Recovery algorithm for short lived flows, as most of the current Internet traffic is. Without modifying the TCP congestion control based on the additive-increase/multiplicative-decrease paradigm, TCP Smart-Framing adopts a novel segmentation algorithm: while classic TCP starts sending one segment, a TCP-SF source is allowed to send an initial window of 4 smaller segments, whose aggregate payload is equal to the connection's MSS. This key idea can be implemented on top of any TCP flavor, from Tahoe to SACK, and requires modifications to the server behavior only.

In this paper we discuss the optimal design of logical topologies in wavelength-routed WDM networks supporting unicast and multicast transfer of IP datagrams. We first explain the key aspects of the problem, emphasizing the fact that in IP networks the routing algorithms are an input to the optimization problem, not an optimization target. We then provide a mixed integer linear programming formulation of the optimization problem., which however leads to unacceptably high complexity for networks of non-trivially small size. We then propose both greedy and metaheuristic approaches for the sub-optimal design of logical topologies with acceptable complexity. Finally, we derive lower bounds that allow the assessment of the performance of the proposed algorithms. Some numerical results indicate that the proposed metaheuristics largely outperform the greedy approaches, and are able to obtain very good logical topologies.

RIO routers, initially proposed in the context of differentiated services networks, distinguish between two classes or packets - in-profile and out-profile - based on a one-bit marking in packet headers. Traditionally, packet-marking is performed by ingress routers according to a user-profile specified in terms of bandwidth. We consider an alternative strategy in which packets are marked based on the state of individual TCP flows. This marking scheme can be implemented either at the end systems or ingress routers (thus avoiding changes at end systems). In addition, we present a variant of our scheme that integrates the Diffserv style packet-marking based on the user-profile. By extensive simulations, we demonstrate that the proposed scheme can significantly reduce the completion time of TCP flows. In particular, the completion time of short TCP flows is halved. These results are consistent over different TCP variants and in the presence of ECN.

This paper describes the network architecture and provides a performance analysis of a passive optical network named SONATA, which has been proposed and demonstrated in the European Union ACTS program. In this nationwide-all-optical network, end-terminals access a single passive routing node via PONs using a TDMA/WDMA access scheme based on slot reservations. The centralized network controller runs resource allocation algorithms to avoid conflicts among end-terminals. Since the resource allocation problem at the network controller can be shown to be in general NP-hard, we provide heuristic algorithms to solve the problem. The analysis of the algorithms is performed via both analysis and simulation.

We develop a general methodology, mainly based upon Lyapunov functions, to derive bounds on average delays, and on queue size averages and variances of complex systems of queues. We then apply this methodology to input-buffered, cell-based switch and router architectures. These architectures require a scheduling algorithm to select at each slot a subset of input-buffered cells which can be transferred towards output ports. Although the stability properties (i.e., the limit throughput) of input-buffered, cell-based switches was already studied for several classes of scheduling algorithms, no analytical results concerning cell delays or queue sizes are yet available in the technical literature. We concentrate on purely input-buffered switches that adopt a maximum weight matching scheduling algorithm, that was proved to be the scheduling algorithm providing the best performance. The derived bounds proved to be rather tight, when compared to simulation results.

We consider cell-based switch and router architectures whose internal switching matrix does not provide enough speed to avoid input buffering. These architectures require a scheduling algorithm to select at each slot a subset of input buffered cells which can be transferred toward output ports. We propose several classes of scheduling algorithms whose stability properties are studied using analytical techniques mainly based upon Lyapunov functions. Original stability conditions are also derived for scheduling algorithms that are being used today in high-performance switch and router architectures.

We consider cell-based switch architectures in which the speedup of the internal switching fabric is not large enough to avoid input buffering. These architectures require a scheduling algorithm to select at each slot a subset of input buffered cells which can be transferred towards output ports. The stability properties of maximal size matching (MSM) scheduling algorithms are studied in the paper, using analytical techniques primarily based upon Lyapunov functions. The main result of the paper is the proof that, for a wide class of MSM scheduling algorithms, stability is guaranteed by an internal switch speedup equal to two.

We consider cell-based switch architectures, whose internal switching matrix does not provide enough speed to avoid input buffering. These architectures require a scheduling algorithm to select at each slot a subset of input buffered cells which can be transferred towards output ports. The stability properties of several classes of scheduling algorithms are studied in the paper, using analytical techniques mainly based upon Lyapunov functions. Original stability conditions are derived for some scheduling algorithms that are being used today in high-performance switch architectures.

This paper describes the network architecture and provides a performance analysis of a passive optical network named SONATA, which has been proposed and demonstrated in the context of the European Union ACTS Program. In this nationwide all-optical network, end terminals access a single passive routing node via PONs using a TDMA/WDMA access scheme based on reservations. The centralized network controller runs resource allocation algorithms in order to avoid conflicts among end terminals. We formally define the resource allocation problem at the network controller, and show that, in general, it is NP-hard. We also provide simple heuristic algorithms to solve the problem. The analysis of the algorithms is performed both via analysis and simulation.

This paper presents a load-balancing method to improve network utilization when static routing algorithms are employed. Static routing algorithms can generally be reduced to a path assignment problem with the aim of minimizing a cost function: i.e., maximizing network revenues. Unfortunately the problem, when multiple constraints such as different QoS traffic classes are present, is computationally intractable. Load-balancing methods are heuristics that allow the identification of local minima of the cost function that hopefully are close to the real optimum. The load-balancing method we propose is based only on static information about the average traffic load of each traffic relation: a piece of information which can easily be attained in telecommunication networks. The path-selection strategy is defined in terms of set theory and its performance is evaluated via simulation by comparison against standard minimum-hop path selections. Simulation results show that low priority traffic benefits significantly from the more balanced load obtained with our solution.