Network-based applications rely on the underlying network infrastructure to reliably forward packets between nodes. The way packets are forwarded has a significant impact on service quality. Therefore, it is important to gain a better understanding of data packet routes. To obtain detailed information about network paths, continuous and long-term packet analysis is required. To achieve this, we present our open-source framework HiPerConTracer 3.0 for large-scale IP trace analysis. It performs Ping and Traceroute measurements to provide detailed insights into packet routes and packet timing by tracing routes between senders and receivers in public and private networks. Particularly, it runs its own measurements, without need to obtain data, or cooperation from, the underlying network service providers or remote server owners. Our tool supports large-scale data collection, storage, and post-processing stages. It supports easy-to-understand route visualization, round-trip time measurements, and hop counts. A proof-of-concept analysis revealed that packet route lengths can change drastically when traveling through unexpected countries, regions, and network operators.

Cloudified mobile networks are expected to deliver a multitude of services with reduced capital and operating expenses. A characteristic example is 5G networks serving several slices in parallel. Such mobile networks, therefore, need to ensure that the SLAs of customised end-to-end sliced services are met. This requires monitoring the resource usage and characteristics of data flows at the virtualised network core, as well as tracking the performance of the radio interfaces and UEs. A centralised monitoring architecture can not scale to support millions of UEs though. This paper, proposes a 2-stage distributed telemetry framework in which UEs act as early warning sensors. After UEs flag an anomaly, a ML model is activated, at network controller, to attribute the cause of the anomaly. The framework achieves 85% F1-score in detecting anomalies caused by different bottlenecks, and an overall 89% F1-score in attributing these bottlenecks. This accuracy of our distributed framework is similar to that of a centralised monitoring system, but with no overhead of transmitting UE-based telemetry data to the centralised controller. The study also finds that passive in-band network telemetry has the potential to replace active monitoring and can further reduce the overhead of a network monitoring system.

Cloud computing is now a ‘go-to’ platform for running various types of application. A wide spectrum of users needing software and hardware resources have embraced the cloud. Following the cloud business model, one is either a cloud-based service provider or a cloud service (hardware and software) user. There is a continuous evolution in the cloud ecosystem to support the ever-changing features of user applications. The cloud ecosystem has added a new resource delegation model to accommodate such new requirements. This new resource delegation model is known as fog. Both, cloud and fog platforms, employ complex hardware and software to provide better run-time support for applications. This paper presents a message-based privacy-aware application execution zone management framework for the cloud/fog platform. The framework aims to create a static execution zone based on the performance of the user application and the data privacy requirements. It also enables the user to set the resource termination conditions. Here, we have prototyped the proposed framework and showed how the proposed approach works with message structure and experiments with a P4 switch. We also present how the prototype could be implemented on a cloud/fog platform.

Cloud computing makes complex processing an off-premise activity by offering software- and hardware-based services using standard security protocols over the Internet. It has been seen that the cloud is not ideal for latency-sensitive applications. Thanks to the current growth of network communication and infrastructure, fog adds a computing resource delegation model between the user and the cloud. Fog aims to improve latency-sensitive applications support. Here, we propose one unified, proactive resource orchestration framework from a cloud/fog service provider perspective. The framework consists of a predictor and a resource allocator module. Users subscribe to these resources to execute their applications. The framework is modular and does not require application-specific information. A service provider can customise each module. We have presented the framework prototype by showing each module's simulated performance results using the parameters of our cloud/fog research testbed.

IT systems monitoring is a crucial process for managing and orchestrating network resources, allowing network providers to rapidly detect and react to most impediment causing network degradation. However, the high growth in size and complexity of current operational networks (2022) demands new solutions to process huge amounts of data (including alarms) reliably and swiftly. Further, as the network becomes progressively more virtualized, the hosting of NFV on cloud environments adds a magnitude of possible bottlenecks outside the control of the service owners. In this paper, we propose two deep learning anomaly detection solutions that leverage service exposure and apply it to automate the detection of service degradation and root cause discovery in a cloudified mobile network that is orchestrated by ETSI OSM. A testbed is built to validate these AI models. The testbed collects monitoring data from the OSM monitoring module, which is then exposed to the external AI anomaly detection modules, tuned to identify the anomalies and the network services causing them. The deep learning solutions are tested using various artificially induced bottlenecks. The AI solutions are shown to correctly detect anomalies and identify the network components involved in the bottlenecks, with certain limitations in a particular type of bottlenecks. A discussion of the right monitoring tools to identify concrete bottlenecks is provided.

In today’s communication-centric world, users generate and exchange a massive amount of data. The Internet helps user data to travel from one part of the world to another, via a complex set of network systems. These systems are intelligent, heterogeneous, and non-transparent to users. This paper presents an extensive, trace-driven study of user data traffic covering five years of observations, six large ISPs, 22 different autonomous systems, and a total of 12 countries. This work aims to make users aware of how their data travels in the Internet, as the interests of ISPs majorly influence the data traffic path. Although data traffic should prefer to travel through countries that share land borders, we found that the shortest land distance between the two countries does not impact data path selection.1

Network Function Virtualization is a key enabler to building future mobile networks in a flexible and cost-efficient way. Such a network is expected to manage and maintain itself with minimum human intervention. With early deployments of the fifth generation of mobile technologies – 5G – around the world, setting up 4G/5G experimental infrastructure is necessary to optimally design Self-Organising Networks (SON). In this demo, we present a custom small-scale 4G/5G testbed. As a step towards self-healing, the testbed integrates Programming Protocol-independent Packet Processors (P4) virtual switches, that are placed along interfaces between different components of transport and core network. This demo not only shows the administration and monitoring of the Evolved Packet Core VNF components, using Open Source MANO, but also serves as a proof of concept for the potential of P4-based telemetry in detecting anomalous behaviour of the mobile network, such as a congestion in the transport part.

The core idea of the Multi-Path Transmission Control Protocol (MPTCP) is to utilize multiple network connections by distributing payload data transmission among several sub-flows. Then, multiple paths in the underlying networks can be used to maximize the overall connection throughput. However, the concurrent transmission on only a subset of all possible sub-flows' aggregation can improve network performance, because of performance differences between the subflow. In this paper, we propose a new FullMesh algorithm based on Path Characteristic and Data Characteristic (PCDC), in which a Subflow Impact Factor (IF) is used as a subflow characteristic to predict the impact of a subflow on the overall throughput. Then, different path sets are adopted for different sizes of traffic. The PCDC algorithm is evaluated in the NORNET CORE testbed, comparing it to the FullMesh algorithm. Our research results show that the PCDC algorithm can improve the network throughput and reduce the overall completion time of small data streams.

Cloud computing has revolutionised the development and deployment of applications by running them cost-effectively in remote data centres. With the increasing need for mobility and micro-services, particularly with the emerging 5G mobile broadband networks, there is also a strong demand for mobile edge computing (MEC). It enables applications to run in small cloud systems in close proximity to the user in order to minimise latencies. Both cloud computing and MEC have their own advantages and disadvantages. Combining these two computing paradigms in a unified multi-cloud platform has the potential of obtaining the best of both worlds. However, a comprehensive study is needed to evaluate the performance gains and the overheads imposed by this combination to real-world cloud applications. In this paper, we introduce a baseline performance evaluation in order to identify the fallacies and pitfalls of combining multiple cloud systems and MEC into a unified MEC-multi-cloud platform. For this purpose, we analyze the basic, application-independent performance metrics of average round-trip time (RTT) and average application payload throughput in a setup consisting of two private and one public cloud systems. This baseline performance analysis confirms the feasibility of MEC-multi-cloud and provides guidelines for designing an autonomic resource provisioning solution in terms of an extension proposed to our existing Melodic middleware platform for multi-cloud applications.

DiffServ was designed to implement service provider quality of service (QoS) policies, where routers change and react upon the DiffServ Code Point (DSCP) in the IP header. However, nowadays, applications are beginning to directly set the DSCP themselves, in the hope that this will yield a more appropriate service for their respective video, audio and data streams. WebRTC is a prime example of such an application. We present measurements, for both IPv4 and IPv6, of what happens to DSCP values along Internet paths after an end system has set them without any prior agreement between a customer and a service provider. We find that the DSCP is often changed or zeroed along the path, but detrimental effects from using the DSCP are extremely rare; moreover, DSCP values sometimes remain intact (potentially having an effect on traffic) for several AS hops. This positive result motivates an analysis of the potential latency impact from such DSCP usage, for which we present the first measurement results. We find that routers at approximately 3% of more than 100,000 links differentiate between the WebRTC DSCP values (EF, AF42 and CS1) and consistently reduce delay in comparison with probes carrying a zero value (CS0) under congestion. In contrast, routers at around 2% of these links increase the delay by a comparable amount under congestion, uniformly for EF, AF42 and CS1.

With the rapid development of Internet communications, there is a growing demand to support devices being connected to multiple Internet service providers simultaneously. For example, every modern smartphone already provides at least mobile broadband (UMTS, LTE) as well as Wi-Fi interfaces. This multi-homing property can be used for resilience, but there is also an increasing interest in making use of concurrent multipath transport. That is, multiple network paths can be utilised simultaneously, in order to improve the payload throughput for applications like big data or cloud computing. In this paper, we examine the performance of multi-path transport in real-world Internet setups, based on Multi-Path TCP (MPTCP) in the NORNET testbed for multi-homed systems. However, systems in such challenging setups need proper configuration. Therefore, we particularly would like to highlight the performance impact of different path management and congestion control settings in such realistic scenarios.

The Multi-Path Transmission Control Protocol (MPTCP) is the new concurrent multi-path transfer extension for the widely-deployed Transmission Control Protocol (TCP). Of course, having multiple and possibly highly dissimilar paths for transmission is a challenge for the management of the send and receive buffers, since optimal throughput is desired with a reasonable allocation of the limited memory resources in MPTCP endpoints. This is particularly important when many MPTCP connections have to be handled simultaneously. This paper measures out the required MPTCP buffer size in the real-world Internet testbed NORNET, comparing theoretical size and real size to analyse MPTCP performance. The experiment shows that multi-path transmission can effectively increase the application payload throughput, and greatly improve the robustness of the data transmission. As an important point of this paper, we can show that appropriate buffer size settings can increase the payload throughput, while not wasting resources. This paper has certain significance for further accurately determining the optimal buffer size settings for multi-path transmission in large-scale Internet setups.

Multipath TCP (MPTCP) enables the simultaneous use of multiple links for bandwidth aggregation, better resource utilization and improved reliability. Its coupled congestion control intends to reap the increased bandwidth of multiple links, while avoiding being more aggressive than regular TCP flows on every used link. We argue that this leads to a very conservative behavior when paths do not share a bottleneck. Therefore, in this paper, we first quantify the penalty of the coupled congestion control for links that do not share a bottleneck. Then, in order to overcome this penalty, we design and implement a practical shared bottleneck detection (SBD) algorithm for MPTCP, namely MPTCP-SBD. Through extensive emulations, we show that MPTCP-SBD outperforms all currently deployed MPTCP coupled congestion controls by accurately detecting bottlenecks. For the non-shared bottleneck scenario, we observe throughput gains of up to 40% with two subflows and the gains increase significantly as the number of subflows increase, reaching more than 100% for five subflows. Furthermore, for the shared bottleneck scenario, we show that MPTCP-SBD remains fair to TCP. We complement the emulation results with real-network experiments justifying its safeness for deployment.

Multi-path transfer protocols such as Concurrent Multi-Path Transfer for SCTP and Multi-Path TCP (MPTCP), are becoming increasingly popular, due to widespread deployment of smartphones with multi-homing support. Although the idea of using multiple interfaces simultaneously to improve application throughput is tempting, does transmission over multiple interfaces always provide benefits especially in realistic setup? In this paper, we first show that multi-path transfer might actually have a negative impact in real-world scenarios with mobile broadband and wireless LAN networks. We then introduce our Dynamic Relative Path Scoring (DRePaS) algorithm that continuously evaluates the contribution of paths to the overall performance and dynamically influences the scheduling decisions to make best use of the paths for the overall system performance. We show that DRePaS outperforms the current MPTCP implementation in terms of throughput and application delay, especially when the links are heterogeneous.

Today, most of the smart phones are equipped with two network interfaces: Mobile Broadband (MBB) and Wireless Local Area Network (WLAN). Multi-path transport protocols provide increased throughput or reliability, by utilizing these interfaces simultaneously. However, multi-path transmission over networks with very different QoS characteristics is a challenge. In this paper, we studied Multi-Path TCP (MPTCP) in heterogeneous networks, specifically MBB networks and WLAN. We first investigate the effect of bufferbloat in MBB on MPTCP performance. Then, we propose a bufferbloat mitigation algorithm: Multi-Path Transport Bufferbloat Mitigation (MPT-BM). Using our algorithm, we conduct experiments in real operational networks. The experimental results show that MPT-BM outperforms the current MPTCP implementation by increasing the application goodput quality and decreasing MPTCP's buffer delay, jitter and buffer space requirements.

VoIP systems, based on the Session Initiation Protocol (SIP), are becoming more and more widespread in the Internet. However, this creates security issues and opens up new opportunities for misuse and fraud. The most widespread threat are multi-stage attacks to commit Toll Fraud. To devise effective countermeasures, it is crucial to know how attacks on these systems are performed in reality. In this paper, we introduce a novel distributed monitoring system with Sensor nodes located in Norway, Germany and China that allow to detect SIP-based attacks from the Internet. Based on experiences from experiments spanning several years, we propose a new setup which allows simple and straightforward addition of new remote observation points. We have deployed this setup in the NorNet testbed and highlight its advantages compared to a previous setup with physically distributed Sensors. We also present results from a 45 day field test with 13 observation points. These results confirm the advantages of a widely distributed monitoring setup and give some new insights into the behavior of the attackers.

Over the last decade, the Internet has grown at a tremendous speed in both size and complexity. Nowadays, a large number of important services – for instance e-commerce, healthcare and many others – depend on the availability of the underlying network. Clearly, service interruptions due to network problems may have a severe impact. On the long way towards the Future Internet, the complexity will grow even further. Therefore, new ideas and concepts must be evaluated thoroughly, and particularly in realistic, real-world Internet scenarios, before they can be deployed for production networks. For this purpose, various testbeds – for instance PlanetLab, GpENI or G-Lab – have been established and are intensively used for research. However, all of these testbeds lack the support for so-called multi-homing. Multi-homing denotes the connection of a site to multiple Internet service providers, in order to achieve redundancy. Clearly, with the need for network availability, there is a steadily growing demand for multi-homing. The idea of the NorNet Core project is to establish a Future Internet research testbed with multi-homed sites, in order to allow researchers to perform experiments with multi-homed systems. Particular use cases for this testbed include realistic experiments in the areas of multi-path routing, load balancing, multi-path transport protocols, overlay networks and network resilience. In this paper, we introduce the NorNet Core testbed as well as its architecture.

Today, a steadily growing number of devices contains multiple network interfaces. For example, nearly all smartphones are equipped with at least W-LAN as well as 3G/4G interfaces. In consequence, there is a rising demand for so-called multi-path transfer, which utilizes all of these interfaces simultaneously in order to maximize the payload throughput of applications. Currently, this so-called multi-path transfer is very actively discussed by the IETF, in form of the Multi-Path TCP (MPTCP) extension for TCP as well as the Concurrent Multi-path Transfer extension for SCTP (CMT-SCTP). Their larger-scale deployment in the Internet is expected for the near future. A key issue that prevents the standardization of these approaches is the fairness to concurrent TCP flows. A multi-path transfer should behave “TCP-friendly”, i.e. cause no harm to the performance of the very widely deployed TCP-based applications. In this paper, we first extend the notion of “fairness” from single-path transport to multi-path transport. Furthermore, we introduce the relevant congestion control approaches in the IETF context for single-path as well as multi-path transfer. We simulatively analyze these approaches in a couple of interesting network configuration scenarios, in order to show their behavior with special regard to the fairness definitions. Particularly, we also point out items of further discussion which are the result of the current approaches.

The classic layered OSI reference model has reached its limits for the Internet of today. In this paper, we propose a clean-slate conceptual design of a new architecture as a contribution to the ongoing discussion on the Future Internet. We address the shortcomings of the layered model by redesigning the classical model. Our approach differs from the concepts found in prior work, which focus on special parts of the problems (such as the application, the service or the event) by staggering back a couple of steps and trying to see the requirements from a different perspective. Our concept - which is denoted as Encapsulated Responsibility-Centric Architecture Model (ERiCA) - focuses on determining the responsibilities by using different planes in addition to a partitioning of the network into different decision domains. With this partitioning, we can reduce the complexity of providing a certain service.

The steadily growing importance of Internet-based applications and their resilience requirements lead to a rising number of multi-homed sites. The idea of Concurrent Multipath Transfer (CMT) is to exploit the existence of multiple paths among endpoints to increase application data throughput. However, handling the congestion control of each path independently lacks of fairness against non-CMT flows. In this paper, we describe our approach of combining CMT with the idea of Resource Pooling (RP) in order to achieve a performance improvement over non-CMT transfer while still remaining fair to concurrent flows on congested links. Unlike existing approaches which adapt classic TCP to a multi-homed CMT protocol, our approach does not depend on specific characteristics of TCP. Instead, we base on already entrenched functional blocks of CMT transfer, on the example of the CMT-enabled SCTP (Stream Control Transmission Protocol). In a simulative proof-of-concept analysis, we show that our approach while relatively simple - is already quite effective.

With the deployment of more and more resilience-critical Internet applications, there is a rising demand for multi-homed network sites. This leads to the desire for simultaneously utilising all available access paths to improve application data throughput. This is commonly known as Concurrent Multipath Transfer (CMT); approaches for several Transport Layer protocols have been proposed. Combined with Resource Pooling (RP), CMT can also fairly coexist with concurrent non-CMT flows. Current approaches focus on symmetric paths (i.e. similar bandwidth, delay and error rate). However, asymmetric paths are much more likely - particularly for realistic Internet setups - and efficient CMT usage on such paths is therefore crucial. In this paper, we first show the challenges of plain as well as RP-aware CMT data transport over asymmetric paths. After that, we introduce mechanisms for efficient transport over such paths. Finally, we analyse the performance of our approaches by using simulations.

With the growing complexity of software applications, there is an increasing demand for solutions to distribute workload into server pools. Grid Computing provides powerful - but also highly complex - mechanisms to realize such tasks. Also, there is a steadily growing number of downtime-critical applications, requiring redundant servers to ensure service availability in case of component failures. To cope with the demand for server redundancy and service availability, the IETF has recently standardized the lightweight Reliable Server Pooling (RSerPool) framework, which is a common architecture for server pool and session management. In this paper, we first introduce the concept of RSerPool and then present the modeling thoughts of RSPLIB and the underlying general groupware design. Based on RSPLIB, we will illustratively show how to easily develop applications on top of RSerPool. We will also offer an application evaluation example for a proof-of-concept setup to distribute ray-tracing computation workload into a compute pool.

The Reliable Server Pooling (RSerPool) architecture is the IETF's new standard for a lightweight server redundancy and session failover framework to support availability-critical applications. RSerPool combines the ideas from different research areas into a single, resource-efficient and unified architecture. Server pools are maintained by redundant management components, which are called registrars. Registrars monitor the availability of servers in the pool and remove them in case of failure. Furthermore, they synchronize their view of the pool with other registrars to provide information redundancy. In this paper, we first analyze the implications of registrar redundancy on the server pool performance. Furthermore, we present an optimization approach for the server pool management, which improves the system performance in case of registrar problems by hardware failures or Denial of Service attacks.

Reliable Server Pooling (RSerPool) is a light-weight protocol framework for server redundancy and session failover, currently still under standardization by the IETF RSerPool WG. While the basic ideas of RSerPool are not new, their combination into a single, resource-efficient and unified architecture is. Server redundancy directly leads to the issues of load distribution and load balancing, which are both important for the performance of RSerPool systems. Therefore, it is crucial to evaluate the performance of such systems with respect to the load balancing strategy required by the application. The goal of our paper is - after presenting a short overview of the RSerPool architecture and its application cases - to provide a quantitative, application-independent performance analysis ofRSerPool's server failure handling capabilities with respect to important adaptive and non- adaptive load balancing strategies. We will also analyse the impact of RSerPool protocol parameters on the performance of the server failure handling functionalities and the network overhead.

In order to provide a generic, application- independent and resource-efficient framework for server redundancy and session failover, the IETF RSerPool WG is currently standardizing the reliable server pooling (RSerPool) framework. Server redundancy has to take load distribution and load balancing into consideration since these issues are crucial for the system performance. There has already been some research on the server selection strategies of RSerPool for different application scenarios. In particular, it has been shown that the adaptive least used selection usually provides the best performance. This strategy requires up-to-date load information of the services, which has to be propagated among distributed pool management components. But network delay (which is realistic for systems being widely distributed to achieve availability in case of regional servers failures) as well as caching of information may both lead to obsolete load information. Therefore, the purpose of this paper is to analyse and evaluate the performance of a new server selection rule to cope with update latencies. Especially, we will also analyse the impact of different workload parameters on the performance of the new server selection strategy.

The Reliable Server Pooling (RSerPool) architecture is the IETFpsilas upcoming standard of a lightweight server redundancy and session failover framework for availability-critical applications. RSerPool combines the ideas from different research areas into a single, resource-efficient and unified architecture. Although there have already been a number of research papers on the pool management, load distribution and failover handling performance of RSerPool, the robustness against intentional attacks has not been intensively addressed yet. Therefore, the first goal of this paper is to provide a robustness analysis in order to outline the attack bandwidth necessary for a significant impact on RSerPool-based services. After that, we present our anomaly detection approach that has been designed to protect RSerPool systems against attacks. We also show the effectiveness of this approach by simulations.

This paper presents the initial evaluation of a novel network device being located in edge nodes. It provides relaxed QoS guarantees to certain flows on a congested link by focussing packet discard on selected flows. In contrast to classical IntServ solutions, our approach requires minimal signalling and therefore provides both efficiency and scalability. In this paper, we first describe the ideas of our QoS device and then provide first results of our ongoing simulative performance evaluation and optimization.

This paper presents a novel network device being located in network edge nodes. It provides a solution for QoS guarantees to certain flows on a congested link by focussing packet discard on selected flows. Unlike IntServ solutions like RSVP, our approach only requires minimal signalling and provides both efficiency and scalability. In this paper, we first describe the ideas of our QoS device and then provide first results from a fast-track simulation model implementing a lightweight version of our approach.

Reliable server pooling (RSerPool) is a protocol framework for server redundancy and session failover, currently under standardization by the IETF RSerPool WG. While the basic ideas of RSerPool are not new, their combination into one architecture is. Some research into the performance of RSerPool for certain specific applications has been made, but a detailed, application-independent sensitivity analysis of the system parameters is still missing. The goal of this paper, after an application-independent, generic quantification of RSerPool systems and definition of performance metrics for both service provider and user, is to systematically investigate RSerPool's behavior on changes of workload and system parameters to give basic guidelines on designing efficient RSerPool systems

In this contribution we present a new type of mobility management for IP-based networks that, contrary to conventional approaches, does not focus on the network layer, but on the transport and session layers. At the heart of this new mobility concept is the reliable transport protocol SCTP, with an enhancement for dynamic address reconfiguration. The reliable server pooling (RSerPool) protocol suite provides a service for session monitoring and control. The suggested solution is transparent for applications, requires no changes in the network infrastructure, and is evaluated with a real-world implementation. Finally, we present first results from the application of this mobility concept to different mobility scenarios. These were obtained from working SCTP and RSerPool implementations that have been developed within our group.

Many Internet services require high availability. Server pooling provides a high availability solution using redundant servers. If one server fails, the service is continued by another one. A challenge for server pooling is efficient state sharing: the new server requires the old one's state to continue service. This paper proposes a simple, efficient and scalable solution, usable for a large subset of applications.