Companies’ Network Operations Centers continuously monitor network health to keep activity fully operational in the current scenario of decentralization of digital workplaces. In this task, network managers have a diverse set of tools to proactively troubleshoot network changes that could potentially lead to network outages. Unfortunately, these tools primarily focus on servers and high-volume services, while changes in end-users’ traffic can also be a symptom of relevant issues such as the misuse and misconfiguration of resources, service slowdowns, and cybersecurity breaches. End-user behavior, in particular, tends to be more erratic and chaotic than server traffic, especially when different users may share the same IP address over time (e.g., DHCP and Wi-Fi environments). To address these challenges, we propose modeling end-user behavior as an unordered collection of activities (i.e., pieces of regular behavior) rather than as time series. These activities, such as downloading a file or browsing the Internet, yield an identifiable sequence of measurements in terms of network metrics over time (e.g., throughput or number of connections). Deviations from typical sequences of measurements are flagged as irregular behaviors, with higher priority assigned to increasingly uncommon patterns. This methodology leverages Markov chains to represent activities as sequences of expert-aware discrete states of network metrics. When applied to a multi-year dataset from a global enterprise, our proposal has been able to cope with the chaotic and challenging environment of end-user behavioral modeling, identifying critical issues such as DNS misconfigurations, compromised printers, and cybersecurity threats. • Users’ behavior is chaotic compared to more predictable servers’ profiles. • Expert system is proposed to identify end-user anomalies. • The system is based on an expert-tailored Markovian approach. • System has been evaluated on a years-long dataset from international company. • This identified DNS misuse, compromised printers, and cybersecurity threats.

Modern data centers require that a Network Operations Center is continuously monitoring network health, desirably in order to take proactive action before potential trouble occurs. In this paper, we contribute to the capacity planning of the workforce in charge. To this end, we have extensively analyzed, with real-world data, behavioral changes in a large server population in a data center. Our findings allow classifying such behavioral changes, which may be indicative of potential trouble, into relevance regions using a ranking mechanism. Then, the proposed methodology allows, together with an estimation of the time to analyze, assessing the workforce necessary to proactively tackle the behavioral changes observed. We conclude with a case study from a working data center, including a hands-on implementation of a traffic analysis solution to detect such behavioral changes and an estimation of the needed workforce to analyze them. Our results show that between 4 and 5 network managers are an adequate number for handling behavioral-changes analysis in a large enterprise data center

Periods of low load have been used for the scheduling of non-interactive tasks since the early stages of computing.Nowadays, the scheduling of bulk transfers-i.e., large-volume transfers without precise timing, such as database distribution, resources replication or backups-stands out among such tasks, given its direct effect on both the performance and billing of networks.Through visual inspection of traffic-demand curves of diverse points of presence (PoP), either a network, link, Internet service provider or Internet exchange point, it becomes apparent that lowuse periods of bandwidth demands occur at early morning, showing a noticeable convex shape.Such observation led us to study and model the time when such demands reach their minimum, on what we have named valley time of a PoP, as an approximation to the ideal moment to carry out bulk transfers.After studying and modeling single-PoP scenarios both temporally and spatially seeking homogeneity in the phenomenon, as well as its extension to multi-PoP scenarios or paths-a meta-PoP constructed as the aggregation of several single PoPs-, we propose a final predictor system for the valley time.This tool works as an oracle for scheduling bulk transfers, with different versions according to time scales and the desired trade-off between precision and complexity.The evaluation of the system, named VTP, has proven its usefulness with errors below an hour on estimating the occurrence of valley times, as well as errors around 10% in terms of bandwidth between the prediction and actual valley traffic.

The relevance of flow-based monitoring in tasks such as the detection of anomalies and denial of use attacks, traffic reporting, performance evaluation, software routing among others motivates the study of the Internet traffic in terms of flows.However, by the time a network manager or practitioner start any of these tasks, they face the challenge of dimensioning probes' capacities, which depends on the number of concurrent flows.Unfortunately, while the bandwidth of a network or link, both in operation and in future deployments, can be known, or at least estimated in advance, nothing is known about the load in terms of concurrent flows.We aimed at filling this gap by studying the concurrence of flows with respect to the bandwidth normalized by factors such as protocol shares, timeouts, applications, time and years among others.As a result, we provide the research community with several models, based on lognormal distributions, and parameter estimates in such a way that any player in the Internet arena can estimate the expectable number of concurrent flows in their own infrastructure.Moreover, such results emerge from a diverse set of network traces so making the extrapolation of conclusions viable.

Online Quality of Service (QoS) assessment in high speed networks is one of the key concerns for service providers, namely to detect QoS degradation on-the-fly as soon as possible and avoid customers’ complaints. In this regard, a Key Performance Indicator (KPI) is the number of TCP retransmissions per flow, which is related to packet losses or increased network and/or client/server latency. However, to accurately detect TCP retransmissions the whole sequence number list should be tracked which is a challenging task in multi-Gb/s networks. In this paper we show that the simplest approach of counting as a retransmission a packet whose sequence number is smaller than the previous one is enough to detect pathological flows with severe retransmissions. Such a lightweight approach eliminates the need of tracking the whole TCP flow history, which severely restricts traffic analysis throughput. Our findings show that low False Positive Rates (FPR) and False Negative Rates (FNR) can be achieved in the detection of such pathological flows with severe retransmissions, which are of paramount importance for QoS monitoring. Most importantly, we show that live detection of such pathological flows at 10 Gb/s rate per processing core is feasible.

The scheduling of bulk transfers such as database distribution, resources replication and security backups, and other non-time-critical tasks has a direct effect on both the performance and cost of a network. We propose to face this problem by studying the valley times - i.e., the minimum use during off-peak periods of a network - as suitable moments to carry out such tasks. To do so, we characterize them considering the valley-hour, which we define as the opposite to the well-known variable busy-hour. Our analysis, based on 6-year-long measurements from 12 points of presence in the Spanish Research and Education Network (RedIRIS), has guided us to model the valley-hour as a Gaussian process. After that, we compare its behavior in different points and detect the main factors that explain its variance, finding significant heterogeneity. With the resulting conclusions, we have proposed a system to predict valley-hours in RedIRIS with errors below an hour for most of the cases.

Network management actions require the retention of data representing the temporal evolution of network state, mainly in the form of time series. Nonetheless, storing and exploiting those measurements is becoming a challenge as the production rate of such data is continuously increasing and data lasting for long time periods are used. To scale up the storage and improve both the analysis and visualization of network measurements, we apply Functional Principal Components Analysis (FPCA) to extract the most meaningful functional features for network time series, pruning those with low informational importance. We compare such algorithm with other state-of-the-art proposals, and show that it achieves lower error for the representation of atypical observations even with higher compression ratios.

Quality of Service (QoS) and Experience (QoE) monitoring is a must during the management of services deployed over the Internet. This is particularly critical for Voice over IP (VoIP), as its degradation is immediately perceived by end users. From our experience, we highlight the impact that TCP segmentation exerts on online VoIP monitoring systems. On the one hand, it makes difficult to interpret the segmented signaling messages for application monitoring. On the other hand, complete messages do not provide information about packet level behavior, which is necessary for internetworking layer monitoring. Paradoxically, the Network Management community has not paid much attention to this fact, although it compromises several VoIP management tasks. To fill in this gap, we provide an empirical evaluation of its impact for the most popular VoIP signaling protocols using traces from enterprise networks, and present the architecture and heuristic that we are currently developing to partially solve the effect of the segmentation. Our proposal avoids the overhead of reconstructing the entire data stream and, at the same time, it enables the analysis of the packets that are actually sent. To do so, it maps TCP flags with segmented application messages, and exploits data structures that reduce latency. In this way, our solution paves the way for online monitoring tools that take into account both internetworking and application layers performance.

Network managers and analysts are well aware of the importance of network traces to understand traffic behavior, detect anomalies, and evaluate performance forensically, among others. However, the storage required for traffic traces has greatly expanded due to increasing network speeds. In this paper, we selectively cap the packet payload to reduce write speed and storage requirements on hard drives and further reduce the computational burden of packet analysis. The proposed techniques take advantage of most packet payloads being useless for analysis purposes, because they are either encrypted or in a proprietary application non-readable format. Conversely, non-ASCII packets from well-known protocols and protocols with some ASCII data are fully captured as they may be potentially useful for network analysis. We have named this approach as selective capping, and we have implemented and integrated it into a high-speed network driver and a software module at user level, to make its operation more transparent and faster to upper layer applications. The results are promising, and selective capping achieves multi-Gb/s rates by exploiting low-level hardware and software techniques to meet the fastest network rates.

In this paper we present an FPGA-based architecture to export flows in 10 Gbps networks, implemented on the NetFPGA-10G platform. Flow-based monitoring is a powerful methodology to analyze and detect network issues, such as congested links or DDoS attacks. Our design provides the following advantages: (i) The architecture allows processing 10 Gbps links without sampling, even for the highest packet rate of 14.88 Mpps (Million packets per second) that corresponds to the shortest (64-byte) Ethernet frames; (ii) It is possible to manage up to 786,432 concurrent flows; (iii) The project is developed in an open-source hardware platform and the HDL code is open to the community; (iv) The proposed approach frees network routers from the burden of exporting flows.

In this paper we present a software-based traffic classification engine running on commodity multi-core hardware, able to process in real-time aggregates of up to 14.2 Mpps over a single 10 Gbps interface -- i.e., the maximum possible packet rate over a 10 Gbps Ethernet links given the minimum frame size of 64 Bytes.

The research community has proved the existence and studied the root causes of Path Inflation on the Internet - end-to-end paths significantly longer than necessary. However, it has been typically ignored that the popularity of traffic destinations and, consequently, of network paths, is clearly heterogeneous - some destinations are popular while others are barely accessed. In this paper, we propose a trace-driven methodology to measure the Path Inflation accounting for the popularity of Internet destinations from a given network, thus evaluating the implications that Path Inflation exerts on real networks under production. This information is important for network operators because it allows them to objectively stand out those destinations whose connection analysis must be prioritized. The results of applying this methodology to the Spanish academic network show that the most critical regions to focus on are Spain's closest countries, which either are very popular or have large Path Inflation as a consequence of the use of transatlantic links as intermediate nodes, or both.

The last few years have witnessed VoIP applications gaining a tremendous popularity and Skype, in particular, is leading this continuous expansion. Unfortunately, Skype follows a closed source and proprietary design, and typically uses encryption mechanisms, making it very difficult to identify its presence from a traffic aggregate. Several algorithms and approaches have been proposed to perform such task with promising results in terms of accuracy. However, such approaches typically require significant computation resources and it is unlikely that they can be deployed in nowadays high-speed networks. In this light, this paper focuses on cutting the processing cost of algorithms to detect Skype traffic. We have conveniently tuned a previous well-validated algorithm and we have assessed its performance. To this end, we have used real traces from public repositories, from a Spanish 3G operator, and synthetic traces. Our results show that a single process can detect Skype traffic at 1 Gbps rates reading replayed real traces directly from a NIC. Even more, 3.7 Gbps are achieved reading from traces previously allocated in memory using a single process and 45 Gbps using 16 concurrent processes. This fact paves the way for 10 Gbps processing in commodity hardware.

The increasing amount of bandwidth requirements and quality of service needs for the next-generation access networks has boosted extensive research in the fiber-optics communication field. In this light, passive optical networks (PONs) combined with optical code division multiple access (OCDMA), provide a potentially cost-effective solution to meet such bandwidth demands. This work proposes an optical transparent architecture which enables all-optical communication between the network nodes. The encoded data streams are multiplexed at a merging point which results in multiple user interference (MUI), thus significantly reducing the network throughput. The networking nodes are able to monitor and record user activity in the PON, and further register the (past) state of activity at the merging point. In this work, we study the coherence of state between the networking nodes and the merging point, for different packet size distributions, in order to predict an optimal transmission instant of each node's data packets. We note that the states are coherent depending on the packet size distribution.

Grid networks have captured a lot of attention in recent years because of their potential to generate new applications thanks to network, computing and storage resources virtualization. Quality of service (QoS) is a key issue for grid services provisioning. Current grid services are provided on multi service networks such as the Internet. Thus, QoS architectures originally developed for the Internet such as DiffServ (DS) have been tested in grid environment. Since grid network services based on Internet networks will be developed in the next years, we propose in this paper to investigate the potentialities of an innovative Internet QoS architecture known as flow-aware networking (FAN). FAN appears as a promising alternative to DS for QoS provisioning in IP networks. DS proceeds to traffic differentiation and QoS provisioning through IP packet marking whereas FAN consists in implicit IP flow differentiation and a flow-based admission control. A grid session may be seen as a succession of parallel TCP flows with voluminous data transfers. In this paper, we compare by means of computer simulations the performance of FAN and DS architectures under grid environment. Two metrics are adopted for that purpose: the average transit delay and the average goodput of a grid session in an IP access router.

In the development of accurate capacity planning and network resource dimensioning models, network operators must handle representative information about the traffic volumes traversing its network. However, the amount of traffic measurements available over which to perform such analysis, processing and storage is overwhelming. For this reason, the research community has understood the importance of finding an effective mechanism to reduce (or subsample) such huge amount of data, with minimum loss of information.

The optical burst switching (OBS) paradigm proposes a new set of transmission protocols and network architectures that permits the high-utilization of the raw bandwidth available by dense wavelength division multiplexing at a moderate computational cost. Accordingly, it is necessary to extend the management plane of such networks to include the particular aspects of OBS, in order to guarantee the appropriate network operation. This paper analyzes the information required for the management of such OBS networks and proposes a new information model based on observed management use-cases. A simple data model is also specified to reduce the computational burden, but compliant with the modeled information.

This work provides a jitter analysis of size-based burst assembly algorithms and also discusses other burst assembly algorithms that use the packet delay as the assembly threshold to provide a bound on jitter.

This papers reports the advances in optical core networks research coordinated in the framework of the e- photon/ONe and e-photon/ONe+ networks of excellence.

Preemption techniques have been recently proposed for service differentiation in optical burst switching (OBS) networks. According to V. Vokkarane et al. (2003), an incoming burst with the same priority that the burst in service will preempt the wavelength if the residual length of the burst in service is smaller than the incoming burst transmission time. For a network scenario with no wavelength conversion, the preemption probability is evaluated assuming exponential, Gaussian and Pareto-distributed burst sizes. Knowledge of the preemption dynamics at an OBS switch is a fundamental issue in performance evaluation, since the downstream switches will surely be affected. An analytical upper bound is provided, that shows that the preemption probability depends on the burst size distribution, which in turn depends on the burst assembly technique used at the network edges. On the other hand, not only truncated bursts result from preemption, as reported in other studies, but also the burst size distribution for preempting bursts is shifted to larger values.

The MinMax scheduling algorithm has been proved to be fair and efficient for dynamic bandwidth allocation in wireless scenarios. In this paper, MinMax is extended to the case of Internet traffic, which is bursty at all timescales. To do so, MinMax is equipped with a minimum square error estimate in order to improve bandwidth allocation accuracy. The proposed MinMax extension is named MinMaxPred, which has been developed following a novel methodology that considers the traffic prediction and bandwidth scheduling problem jointly while, traditionally, they were considered as isolated problems.

The blocking time distribution for an OBS router is obtained, under the assumption of Poisson-arriving bursts with Pareto, Gaussian and exponential burst size distributions. Analytical expressions are provided as a function of number of wavelengths per port. Such expressions can be used to dimension fiber delay lines (FDLs) and to perform end-to-end delay estimation. On the other hand, we show that the blocking time distribution becomes exponential as the number of wavelengths increases, regardless of the burst size distribution. Since the burst size distribution is determined by the burst assembly algorithm at the network edges, we conclude that the burst assembly algorithm have no influence on both burst blocking probability and burst blocking time in future DWDM networks.

The use of IP traffic prediction techniques for DAMA TDMA protocols is investigated in this paper. The predicted traffic distribution is derived when the input traffic shows long-range dependence features. Furthermore, an equivalent bandwidth is calculated, which allows the wireless terminal to request a certain amount of bandwidth (slot duration) in terms of a target traffic loss probability. The numerical results indicate very good traffic prediction capabilities, together with moderate bandwidth loss.

We show that prediction algorithms in the least mean square error sense prove better in a burst rather than in a packet switching network. For the latter, further information about the packet arrival distribution within the prediction interval is required. Regarding burst switching, we compare optical burst switching networks with and without linear prediction to conclude that linear prediction provides a significant improvement in end-to-end latency with low bandwidth waste.

A commonly accepted traffic model for a large population of Internet users consists of a multiplex of Poisson-arriving heavy-tailed streams with the same constant rate (M/G//spl infin/). We show that even though such a regular model provides an accurate description of long-range dependence, the marginal distribution variance is underestimated, resulting in erroneous calculation of overflow probability in network simulations. On the other hand, we show that the traffic variability due to the marginal distribution variance can be the limiting factor for performance in the gigabit-speed next-generation Internet, rather than the long-range dependence features present in today's traffic.