This paper introduces ZEROTuNE, a novel cost model for parallel and distributed stream processing that can be used to effectively set initial parallelism degrees of streaming queries. Unlike existing models, which rely majorly on online learning statistics that are non-transferable, context-specific, and require extensive training, ZEROTuNE proposes data-efficient zero-shot learning techniques that enable very accurate cost predictions without having observed any query deployment. To overcome these challenges, we propose ZEROTuNE, a graph neural network architecture that can learn from the structural complexity of parallel distributed stream processing systems, enabling them to adapt to unseen workloads and hardware configurations. In our experiments, we show when integrating ZEROTuNE in a distributed streaming system such as Apache Flink, we can accurately set the degree of parallelism, showing an average speed-up of around 5× in comparison to existing approaches.

Although privacy-preserving mechanisms endeavor to safeguard sensitive information at the attribute level, detected event patterns can still disclose privacy-sensitive knowledge in distributed complex event processing systems (DCEP). Events might not be inherently sensitive, but their aggregation into a pattern could still breach privacy. In this paper, we study in the context of APP-CEP the problem of integrating pattern-level privacy in event-based systems by selective assignment of obfuscation techniques to conceal private information. Compared to state-of-the-art techniques, we seek to enforce privacy independent of the actual events in streams. To support this, we acquire queries and privacy requirements using CEP-like patterns. The protection of privacy is accomplished through generating pattern dependency graphs, leading to dynamically appointing those techniques that have no consequences on detecting other sensitive patterns, as well as non-sensitive patterns required to provide acceptable Quality of Service. Besides, we model the knowledge that might be possessed by potential adversaries to violate privacy and its impacts on the obfuscation procedure. We assessed the performance of APP-CEP in a real-world scenario involving an online retailer’s transactions. Our evaluation results demonstrate that APP-CEP successfully provides a privacy-utility trade-off. Modeling the background knowledge also effectively prevents adversaries from realizing the modifications in the input streams.

In this paper we propose P4-based Atomic Multicast (P4mCast), a new in-network atomic multicast protocol to support total order guarantees for State Machine Replication (SMR) in cloud-based fault-tolerant and distributed applications. P4mCast builds on in-network computing, applying leader-based consensus to groups of prominent${P 4}$programmable switches in modern data center networks. P4mCast achieves significantly lower latency overhead in the microseconds scale while increasing the throughput one order of magnitude higher compared to state of the art software-based solutions.

Current network functions consume a significant amount of energy and lack the capacity to support more expressive learning models like neuromorphic functions. The major reason is the underlying transistor-based components that require continuous energy-intensive data movements between the storage and computational units. In this research, we propose the use of a novel component, called Memristor, which can colocalize computation and storage, and provide computational capabilities. Building on memristors, we propose the concept of match-compute processing for supporting energy efficient network functions. Considering the analog processing of memristors, we propose a Probabilistic Content Addressable Memory (pCAM) abstraction which can provide analog match functions. pCAM provides deterministic and probabilistic outputs depending upon the closeness of match of an incoming query with the specified network policy. pCAM uses a crossbar array for line rate matrix multiplications on the match outputs. We proposed a match-compute packet processing architecture and developed the programming abstractions for a baseline network function, i.e., Active Queue Management, which drops packets based upon the higher-order derivatives of sojourn times and buffer sizes. The analysis of match-compute processing over a physically fabricated memristor chip showed only 0.01 fJ/bit/cell of energy consumption, which is 50 times less than the traditional match-action processing.

Active Queue Management (AQM) algorithms are heavily used in packet processors to maintain an optimal queue size and avoid issues like Bufferbloat. Despite the remarkable performance, the traditional AQM algorithms face a major challenge of estimating the accurate queue congestion due to bursty network conditions. The major reason is the use of baseline queue statistics for congestion estimation like delay and sojourn time for Random Early Detection (RED) and Controlled Delay (CoDel), respectively. In this paper, we propose a novel dAQM algorithm that uses advanced traffic statistics like three higher-order derivatives of sojourn time and buffer size along with the baseline sojourn time and buffer size for accurate congestion estimation. dAQM adjusts its drop rate based on the continuously varying congestion to cater to the needs of bursty traffic. We simulated dAQM in ns-3 and analyzed its performance for FTP traffic by variation in traffic load and packet sizes. The results showed that dAQM provides at least 25% and 39.7% reduction in packet loss ratio and flow completion time, respectively, as compared to the traditional AQM algorithms.

Time-Sensitive Networking (TSN) enables deterministic and low-latency communication for real-time applications over Ethernet. That is accomplished by leveraging scheduling and shaping techniques configured for each egress port within the network switches. Although Time Aware Shaper (TAS) is a promising solution for TSN, its adoption often involves substantial complexity. In this work, we propose Residence Delay Aggregation (RDA), a novel asynchronous TSN mechanism that offers dynamic traffic scheduling adapted to the traffic load. Specifically, the proposed RDA mechanism provides upper bound delays similar to other asynchronous TSN mechanisms while improving the flexibility of traffic scheduling and reducing the deployment complexity.

Current network functions build heavily on fixed programmed rules and lack capacity to support more expressive learning models, e.g. brain-inspired Cognitive computational models using neuromorphic computations. The major reason for this shortcoming is the huge energy consumption and limitation in expressiveness by the underlying TCAM-based digital packet processors. In this research, we show that recent emerging technologies from the analog domain have a high potential in supporting network functions with energy efficiency and more expressiveness, so called cognitive functions. We propose an analog packet processing architecture building on a novel technology named Memristors. We develop a novel analog match-action memory called Probabilistic Content-Addressable Memory (pCAM) for supporting deterministic and probabilistic match functions. We develop the programming abstractions and show the support of pCAM for an active queue management-based analog network function. The analysis over an experimental dataset of a memristor chip showed only 0.01 fJ/bit/cell of energy consumption for corresponding analog computations which is 50 times less than digital computations.

We investigate the application of a recently published metric for flexibility in the context of combined port queue schedules of network paths in Time-Sensitive Networks (TSN). TSN comprises a set of specifications for deterministic networking, including support for scheduled traffic with guaranteed deterministic end-to-end delays. Typically, scheduler resource allocation in TSN disregards flexibility of scheduler configurations. Essentially, the notion of flexibility of paths comprising multiple concatenated ports having each a TSN configuration is based on the number of possible embeddings, i.e., resource allocations, for a new flow of a given specification (size and delay deadline) along that path. This demonstration allows the user to define TSN schedules along network paths and, hence, illustrates the behavior and benefit of performing flexibility-aware TSN configuration.

This paper investigates how to complement Complex Event Processing (CEP) with dynamic quality monitoring mechanisms and support the dynamic integration of suitable sensory data sources. In the proposed approach, queries to detect complex events are annotated with consumer-definable quality policies that are evaluated and used to autonomously assign (or even configure) suitable data sources of the sensing infrastructure. We present and study different forms of expressing quality policies and explore how they affect the process of quality monitoring including different modes of assessing and applying quality-related adaptations. A performance study in an IoT scenario shows that the proposed mechanisms in supporting quality policy monitoring and adaptively selecting suitable data sources succeed in enhancing the acquired quality of results while fulfilling consumers' quality requirements. We show that the quality-based selection of sensor sources also extends the network's lifetime by optimizing the data sources' energy consumption.

Deep learning (DL) inference has become an essential building block in modern intelligent applications. Due to the high computational intensity of DL, it is critical to scale DL inference serving systems in response to fluctuating workloads to achieve resource efficiency. Meanwhile, intelligent applications often require strict service level agreements (SLAs), which need to be guaranteed when the system is scaled. The problem is complex and has been tackled only in simple scenarios so far.This paper describes FA2, a fast and accurate autoscaler concept for DL inference serving systems. In contrast to related works, FA2 adopts a general, contrived two-phase approach. Specifically, it starts by capturing the autoscaling challenges in a comprehensive graph-based model. Then, FA2 applies targeted graph transformation and makes autoscaling decisions with an efficient algorithm based on dynamic programming. We implemented FA2 and built and evaluated a prototype. Compared with state-of-the-art autoscaling solutions, our experiments showed FA2 to achieve significant resource reduction (19% under CPUs and 25% under GPUs, on average) in combination with low SLA violations (less than 1.5%). FA2 performed close to the theoretical optimum, matching exactly the optimal decisions (with the least required resources) in 96.8% of all the cases in our evaluation.

Fixed buffer sizing in computer networks, especially the Internet, is a compromise between latency and bandwidth. A decision in favor of high bandwidth, implying larger buffers, subordinates the latency as a consequence of constantly filled buffers. This phenomenon is called Bufferbloat. Active Queue Management (AQM) algorithms such as CoDel or PIE, designed for the use on software based hosts, offer a flow agnostic remedy to Bufferbloat by controlling the queue filling and hence the latency through subtle packet drops.In previous work, we have shown that the data plane programming language P4 is powerful enough to implement the CoDel algorithm. While legacy software algorithms can be easily compiled onto almost any processing architecture, this is not generally true for AQM on programmable data plane hardware, i.e., programmable packet processors. In this work, we highlight corresponding challenges, demonstrate how to tackle them, and provide techniques enabling the implementation of such AQM algorithms on different high speed P4-programmable data plane hardware targets. In addition, we provide measurement results created on different P4-programmable data plane targets. The resulting latency measurements reveal the feasibility and the constraints to be considered to perform Active Queue Management within these devices. Finally, we release the source code and instructions to reproduce the results in this paper as open source to the research community.

Time-Sensitive Networking emerged as a convergent Ethernet-based real-time networking standard for industrial applications. To support real-time, jitter-free isochronous traffic the corresponding TSN mechanism denoted Time Aware Shaper requires special hardware support. In this work, we propose a path to building TSN networks on top of programmable switches. Specifically, we show here how to leverage a data structure amenable to programmable data planes known as Push-in First-out (PIFO) queue to support TSN traffic scheduling for isochronous real-time, as well as, best effort traffic.

In Time-Sensitive Networks (TSN) applications with the highest real-time flow requirements are deployed using the Time-Aware Shaper which requires careful planning and scheduling of flows before deployment. Such deployments lack support for dynamic industrial scenarios such as modular machine assembly and reconfiguration, which require a flexible transition between real-time tasks. In contrast, state-of-the-art techniques rely on flow rescheduling and deployment in conjunction with undesired network downtime. Existing works on adapting schedules to traffic admissions are limited in their ability to choose suitable flows to account for future tasks. In this paper, we aim to leverage the flexibility of scheduler configurations to enable TSN dynamic reconfigurability at runtime. We propose a notion of flexibility for TSN Time-Aware Shaper schedules which we utilize to decide the admissibility of consecutive real-time tasks.

Complex Event Processing (CEP) is a powerful paradigm for scalable data management that is employed in many real-world scenarios such as detecting credit card fraud in banks. The so-called complex events are expressed using a specification language that is typically implemented and executed on a specific runtime system. While the tight coupling of these two components has been regarded as the key for supporting CEP at high performance, such dependencies pose several inherent challenges as follows. (1) Application development atop a CEP system requires extensive knowledge of how the runtime system operates, which is typically highly complex in nature. (2) The specification language dependence requires the need of domain experts and further restricts and steepens the learning curve for application developers.In this paper, we propose CEPless, a scalable data management system that decouples the specification from the runtime system by building on the principles of serverless computing. CEPless provides "operator as a service" and offers flexibility by enabling the development of CEP application in any specification language while abstracting away the complexity of the CEP runtime system. As part of CEPless, we designed and evaluated novel mechanisms for in-memory processing and batching that enable the stateful processing of CEP operators even under high rates of ingested events. Our evaluation demonstrates that CEPless can be easily integrated into existing CEP systems like Apache Flink while attaining similar throughput under high scale of events (up to 100K events per second) and dynamic operator update in ~238 ms.

Publish/subscribe systems have to react fast on changes in their environment while handling many events with low end-to-end latency and high throughput. Moving the broker functionality of publish/subscribe systems to the underlying network layer reduces the path length of events and, in addition, forwarding benefits from powerful and programmable hardware. So far attempts of underlay publish/subscribe depend on a specific API of the network devices, e. g., the OpenFlow protocol, which have restrictions in dealing with dynamic devices and corresponding changes in the introduced attribute names for matching and filtering events.In this work, we focus on the next generation of network devices, which are envisioned to provide reconfigurable hardware components, specified by the open P4 description language. We introduce two new approaches that enable a flexible and generic attribute/value encoding, understandable by P4-capable packet processors, to benefit from the performance properties of hardware. Furthermore, the proposed approaches reduce the effort in encoding and decoding event messages.

Many software based traffic load generators suffer from packet rate variation which is known as rate jitter. In this Demo, we show how this varying rate burstiness can affect the device under test even if the generated average data rate seems constant. To this end, we compare a hardware rate shaping, which is implemented using a programmable P4-switch, and a conventional software load generator and show their impact on a software device under test. The results show, that microbursts within the test load significantly impact the experiment results. Our recommendation is to benchmark the traffic load generator before conducting measurement experiments especially when the device under test is sensitive to microbursts.

QoS requirements of current network control and management applications require the ability to conduct precise measurements of network elements, including switches, routers and Virtual Network Functions (VNFs). State-of-the-art network switches have a forwarding delay of 1µs and below and offer high bandwidths of hundreds Gigabits per second. This imposes high time accuracy and loss-detection requirements on measurement equipment that are not met by existing, software-based measurement tools. The use of specialized tools, meeting these requirements, is restricted by limited flexibility and high cost.In this work, we introduce P4STA, an open source frame-work that combines the flexibility of software-based traffic load generation with the accuracy of hardware packet timestamping. Our evaluation results, obtained using an off-the-shelf P4-programmable switch, show that a time resolution up to 1ns can be achieved on these programmable data plane platforms. Moreover we show how to combine the traffic load of multiple software-based load generators to achieve a measurement load of up to 100Gbit/s per port. Experiments on further programmable platforms, specifically on P4-SmartNICs and FPGAs, show similar results. With this work, we make P4STA available for the research community to advance high performance experiment measurements at nanosecond accuracy.

Driven by real-time applications such as IIoT, TSN and vehicular networks, the optimization of networks and its elements regarding latency and throughput becomes more and more important. With this demo we show how latencies of network components can be identified within nanosecond accuracy by use of commodity P4 hardware. We show a measured propagation speed of$5ns/m$in fiber optical cables. Besides that, our approach scales up to$100Gbit/s$link speed by the aggregation of many low-cost load generators to a flexible software-based load generation.

Emerging network architectures like Information-Centric Networking (ICN)offer simplicity in the data plane by addressing named data. Such flexibility opens up the possibility to move data processing inside network elements for high-performance computation, known as in-network processing. However, existing ICN architectures are limited in terms of (i)in-network processing and (ii)data plane programming abstractions. Such architectures can benefit from Complex Event Processing (CEP), an in-network processing paradigm to efficiently process data inside the data plane. Yet, it is extremely challenging to integrate CEP because the current communication model of ICN is limited to consumer-initiated interaction that comes with significant overhead in number of requests to process continuous data streams. In contrast, a change to producer-initiated interaction, as favored by CEP, imposes severe limitations for request-reply interactions. In this paper, we propose an in-network CEP architecture, INETCEP that supports unified interaction patterns (consumer- and producer-initiated). In addition, we provide a CEP query language and facilitate CEP operations while increasing the range of applications that can be supported by ICN. We provide an open source implementation and evaluation of INETCEP over an ICN architecture, Named Function Networking, and two applications: energy forecasting in smart homes and a disaster scenario.

Large-scale telecommunications providers have to continuously challenge and evolve their network infrastructure to efficiently serve growing markets demands. They must increase performance, lower time-to-market, provide new services, and lower the cost of the infrastructure and its operation. Network Functions Virtualization (NFV) on commodity hardware offers an attractive, low-cost platform to establish innovations much faster than with purpose-built hardware products. Unfortunately, implementing NFV on commodity processors does not match the performance requirements of the high-throughput data plane components in large carrier access networks. In this article, we propose a way to offer residential network access with programmable packet processing architectures. Based on the highly flexible P4 programming language, we present a design and open source implementation of a BNG data plane that meets the challenging demands of Broadband Network Gateways in carrier-grade environments. The proposed evaluation results show the desired performance characteristics and our proposed design together with upcoming P4 hardware can offer a giant leap towards highest performance NFV network access.

Countless novel approaches to communication protocols, overlay networks, and distributed middleware are published every year, yet the adoption of such novel findings in the global Internet landscape progresses at a slow pace. Many of such new communication mechanisms excel (only) under specific deployment conditions, while user mobility and application usage patterns lead to dynamic operation conditions. This mismatch is one reason that makes a wide deployment of new specialized mechanisms particularly hard as observed, for example, for multipath transport protocol extensions until the emergence of multipath transmission control protocol (TCP). This paper formalizes the concept of Transitions, i.e., a method to instrumentalize adaptivity at runtime in communication systems. It allows to exchange communication mechanisms in a running system to optimize the communication quality. In the following, we describe the building blocks required to: 1) capture the features and relations within a communication system and 2) express and optimize the decision making process in such a system. We show how this concept maps intuitively to the Internet model which makes a protocol-independent deployment of applications feasible in the future Internet.

In this special issue, we have collected and presented recent works on innovative approaches and emerged technologies for coping with dynamicity, heterogeneity, and the scale, which have been central to (or even enablers of) recent advances in communications and networking technologies. At a time of an ever-increasing demand for networking resources and a larger scale, communication networks have faced challenges due to the heterogeneity of the demands, the diversity of communication mechanisms, the high dynamicity of the environments, the virtualization of functions, and the stringent and dynamic quality requirements. In recent years, there have been notable advancements in research and development of concepts and methods for highly adaptive and scalable communication networks.This special issue focuses on recent advances in the field of adaptive and scalable communications.

This paper presents MACI, the first bespoke framework for the management, the scalable execution, and the interactive analysis of a large number of network experiments. Driven by the desire to avoid repetitive implementation of just a few scripts for the execution and analysis of experiments, MACI emerged as a generic framework for network experiments that significantly increases efficiency and ensures reproducibility. MACI incorporates and integrates established simulators and analysis tools to foster rapid but systematic network experiments.

Multipath TCP (MPTCP) is a recent TCP evolution that uses multiple TCP subflows and thereby different network paths for a single MPTCP connection. The MPTCP scheduler has a significant impact on the overall performance of MPTCP, as it maps outgoing packets on TCP subflows. In this paper, we identify limitations of today's MPTCP schedulers for thin streams. We present a novel MPTCP scheduler which overcomes today's limitations for thin streams by using i) active probing of unused subflows and ii) timely one-way delay information. A systematic evaluation within our MPTCP Linux Kernel implementation shows that our novel scheduler outperforms established MPTCP schedulers with regard to application-layer round-trip time without sacrificing efficiency and throughput.

QUIC is the emerging transport layer protocol, providing encrypted, stream-multiplexed, low-latency data transfer. In this paper, we propose multipath-enabled QUIC (MPQUIC) to leverage multiple network interfaces, such as WiFi and LTE on today's mobile devices. We show how our MPQUIC design conceptually evolves beyond existing multipathing protocols, such as MPTCP, as it provides fine-grained stream-to-path scheduling, reduced head-of-line blocking, and faster subflow establishment. We present an userland implementation of MPQUIC that is deployable without operating system changes. Our evaluation results show that MPQUIC increases throughput in comparison to traditional QUIC, TCP and even the currently de facto multipath transport protocol MPTCP. First real world measurements confirm that MPQUIC is deployable in the Internet to reduce download times. Moreover, we show that MPQUIC's conceptual advantages over MPTCP efficiently reduce head-of-line blocking in heterogeneous environments. With multipathing support, QUIC is ready to become the universal stream transport protocol in today's Internet.

Mobile applications increasingly rely on frequent and accurate position updates-e.g., with GPS-or Wi-Fi-assisted localization techniques-to provide for functionality to their users. The service quality and acceptance of the application depend strongly on the localization accuracy and the introduced costs, in form of the resource consumption, of the used localization technique. Current mechanisms for location retrieval, however, are limited to non-mobile scenarios or still introduce high costs while obtaining the location. In this work, we propose a collaborative location retrieval service for location-based services in mobile scenarios that combines the location information of a subset of users with the connectivity information between users to enable accurate and cost-efficient location estimations. We evaluate a prototype of our solution to study the impact of service compositions in changing environments and to assess the potential of our proposed service compared to the current state-of-the-art used within location-based services. Our results reveal that, depending on the localization technique, the costs can be reduced significantly while the achieved sensing accuracy and fairness among users improves strongly at the same time.

The advent of the Internet of Things (IoT), with modern sensors and sensor-based devices, will significantly stimulate the development of context-aware applications. An effective means to extract higher-level contextual information from sensor data is distributed complex event processing (CEP), which facilitates the analysis of real-time data streams coming from heterogeneous and distributed sources. Considering that user context is inherently sensitive information, the preservation of privacy is critical once the processing of user context takes place over several (possibly malicious) devices, especially in collaborative scenarios. In this paper, we tackle this issue by introducing a trust-based approach for the placement and execution of CEP operators in a distributed environment. We propose a trust management model based on communication interactions among the users. Furthermore, we incorporate trust recommendations using a cosine-based similarity check in order to overcome collusion and on-off attacks. We developed a smartphone-based distributed CEP system called TrustCEP to evaluate our approach for trust management. Based on the evaluation of TrustCEP, we observe that our approach induces a minimal increase in average battery consumption compared to privacy-negligent approaches.

Multipath TCP enables remarkable optimizations for throughput, load balancing, and mobility in today's networks. The design space of Multipath TCP scheduling, i.e., the application-aware mapping of packets to paths, is largely unexplored due to its inherent complexity. Evidence in this paper suggests that an application-aware scheduling decision, if leveraged right, pushes Multipath TCP beyond throughput optimization and thereby provides benefits for a wide range of applications.

With the increasing popularity of software-defined networking (SDN), ternary content-addressable memory of switches can be directly accessed by a publish/subscribe middleware to perform filtering operations at low latency. In this way, three important requirements for a publish/subscribe middleware can be fulfilled, namely, bandwidth efficiency, line-rate performance, and low latency in forwarding messages between producers and consumers. Nevertheless, it is challenging to sustain line-rate performance in the presence of dynamically changing interests of producers and consumers. In this paper, we realize a scalable, SDN-based publish/subscribe middleware, called PLEROMA, that performs efficient forwarding at line-rate. Moreover, PLEROMA offers methods to efficiently reconfigure a deployed topology in the presence of dynamic subscriptions and advertisements. We evaluate the performance of both the data plane and the control plane of PLEROMA to support our claim. Furthermore, we evaluate and benchmark the performances of SDN-compliant hardware and software switches in the context of our middleware.

With a plethora of sensors and ubiquitous access to the Internet, modern smartphones have enabled a broad range of context-based applications. Most applications make use of the user's physical location to filter relevant content. However, filtering based on dynamic contextual information results in high complexity of the filtering process. This limits the applicability of existing publish/subscribe systems, as they rely on aggregation of filters and fast decentralized matching and forwarding. In this work, we propose a mechanism for transitions between different filter schemes for location-based services. Our mechanism adapts the filtering process to the dynamics in user behavior and resulting load by trading computational complexity at the broker against communication overhead and computational complexity at the mobile client. We integrate our mechanism into an existing publish/subscribe system and evaluate transitions between a context-based filter scheme and two channel-based filter schemes, showing the applicability of our approach.

The tremendous number of sensors and smart objects being deployed in the Internet of Things (IoT) pose the potential for IT systems to detect and react to live-situations. For using this hidden potential, complex event processing (CEP) systems offer means to efficiently detect event patterns (complex events) in the sensor streams and therefore, help in realizing a “distributed intelligence” in the IoT. With the increasing number of data sources and the increasing volume at which data is produced, parallelization of event detection is crucial to limit the time events need to be buffered before they actually can be processed. In this paper, we propose a pattern-sensitive partitioning model for data streams that is capable of achieving a high degree of parallelism in detecting event patterns, which formerly could only consistently be detected in a sequential manner or at a low parallelization degree. Moreover, we propose methods to dynamically adapt the parallelization degree to limit the buffering imposed on event detection in the presence of dynamic changes to the workload. Extensive evaluations of the system behavior show that the proposed partitioning model allows for a high degree of parallelism and that the proposed adaptation methods are able to meet a buffering limit for event detection under high and dynamic workloads.

Complex Event Processing (CEP) systems enable applications to react to live-situations by detecting event patterns (complex events) in data streams. With the increasing number of data sources and the increasing volume at which data is produced, parallelization of event detection is becoming of tremendous importance to limit the time events need to be buffered before they actually can be processed by an event detector - named event processing operator. In this paper, we propose a pattern-sensitive partitioning model for data streams that is capable of achieving a high degree of parallelism for event patterns which formerly could only be consistently detected in a sequential manner or at a low parallelization degree. Moreover, we propose methods to dynamically adapt the parallelization degree to limit the buffering imposed on event detection in the presence of dynamic changes to the workload. Extensive evaluations of the system behavior show that the proposed partitioning model allows for a high degree of parallelism and that the proposed adaptation methods are able to meet the buffering level for event detection under high and dynamic workloads.

With the increasing popularity of Software-defined networks (SDN), TCAM memory of switches can be directly accessed by a publish/subscribe middleware to perform filtering operations at low latency. This way two important requirements for a publish/subscribe middleware can be fulfilled: namely bandwidth efficiency and line-rate performance in forwarding messages between producers and consumers. Nevertheless, it is challenging to sustain line-rate performance in the presence of dynamic changes in the interest of producers and consumers. In this paper, we propose and evaluate the PLEROMA middleware to realize publish/subscribe at line-rate and bandwidth efficiently in SDN. PLEROMA offers methods to efficiently reconfigure a deployed topology in the presence of dynamic subscriptions and advertisements. Furthermore, PLEROMA ensures interoperability and independent reconfiguration of multiple controlled SDN networks.

With the proliferation of mobile devices and sensors, complex event proceesing (CEP) is becoming increasingly important to scalably detect situations in real time. Current CEP systems are not capable of dealing efficiently with highly dynamic mobile consumers whose interests change with their location. We introduce the distributed mobile CEP (MCEP) system which automatically adapts the processing of events according to a consumer's location. MCEP significantly reduces latency, network utilization, and processing overhead by providing on-demand and opportunistic adaptation algorithms to dynamically assign event streams and computing resources to operators of the MCEP system.

The provisioning of basic security mechanisms such as authentication and confidentiality is highly challenging in a content-based publish/subscribe system. Authentication of publishers and subscribers is difficult to achieve due to the loose coupling of publishers and subscribers. Likewise, confidentiality of events and subscriptions conflicts with content-based routing. This paper presents a novel approach to provide confidentiality and authentication in a broker-less content-based publish/subscribe system. The authentication of publishers and subscribers as well as confidentiality of events is ensured, by adapting the pairing-based cryptography mechanisms, to the needs of a publish/subscribe system. Furthermore, an algorithm to cluster subscribers according to their subscriptions preserves a weak notion of subscription confidentiality. In addition to our previous work , this paper contributes 1) use of searchable encryption to enable efficient routing of encrypted events, 2) multicredential routing a new event dissemination strategy to strengthen the weak subscription confidentiality, and 3) thorough analysis of different attacks on subscription confidentiality. The overall approach provides fine-grained key management and the cost for encryption, decryption, and routing is in the order of subscribed attributes. Moreover, the evaluations show that providing security is affordable w.r.t. 1) throughput of the proposed cryptographic primitives, and 2) delays incurred during the construction of the publish/subscribe overlay and the event dissemination.

SUMMARY Group communication is deployed in many evolving Internet‐scale cooperative applications such as multiplayer online games and virtual worlds to efficiently support interaction on information relevant to a potentially very large number of users or objects. Especially peer‐to‐peer based group communication protocols have evolved as a promising approach to allow intercommunication between many distributed peers. Yet, the delivery semantics of robust and scalable protocols such as gossiping is not sufficient to support consistency semantics beyond eventual consistency because no relationship on the order of events is enforced. On the other hand, traditional consistency models provided by reliable group communication providing causal or even total order are restricted to support only small groups. This article proposes thecluster consistencymodel which bridges the gap between traditional and current approaches in supporting both scalability and ordered event delivery. We introduce a dynamic and fault tolerant cluster management method that can coordinate concurrent access to resources in a peer‐to‐peer system and can be used to establishfault‐tolerantconfigurable cluster consistency with predictable reliability, running on top of decentralised probabilistic protocols supporting scalable group communication. This is achieved by a general two‐layered architecture that can be applied on top of the standard Internet communication layers and offers a modular, layered set of services to the applications that need them. Further, we present afault‐tolerantmethod implementing causal cluster consistency with predictable reliability, running on top of decentralised probabilistic protocols supporting group communication. This paper provides analytical and experimental evaluation of the properties regarding the fault tolerance of the approach. Furthermore, our experimental study, conducted by implementing and evaluating the two‐layered architecture on top of standard Internet transport services, shows that the approach scales well, imposes an even load on the system, and provides high‐probability reliability guarantees. Copyright © 2011 John Wiley & Sons, Ltd.

Complex Event Processing (CEP) is of increasing importance in many industrial applications to integrate a huge number of events in a scalable manner. A core challenge towards scalable CEP is to efficiently distribute the rules which define how correlations between events can be detected within an event processing network. Although significant progress has been made recently, there remains a fundamental gap in supporting requirements that emerge from deploying CEP over heterogeneous and independent processing environments. Heterogeneity typically imposes many constraints on the placement of rules, which increases the complexity of the underlying optimization problem and cannot be handled efficiently by existing solutions. In this paper we examine the distributed placement, migration and optimization of rules in the context of the constraint optimization problem to minimize network usage. We propose and evaluate a placement algorithm that efficiently finds valid solutions in scenarios where the solution space is heavily restricted by constraints. The algorithm operates in a decentralized way and is adaptive to dynamic changes of processing nodes, rules, and load characteristics of the event processing network. The proposed rule migration policies resolve invalid placements quickly and thus ensure high availability. The evaluations show that the proposed algorithm is able to efficiently find near optimum solutions within heavy constraint-driven network conditions.

Many application classes such as monitoring applications, involve processing a massive amount of data from a possibly huge number of data sources. Complex Event Processing (CEP) has evolved as the paradigm of choice to determine meaningful situations (complex events) by performing stepwise correlation over event streams. To keep up with the high scalability demands of growing input streams, recent approaches distribute event correlation over several correlation nodes. However, already a failure of a single correlation node impacts the correctness of the final correlation result. In this paper, we illustrate the importance of a strong reliability semantics for CEP in the context of a monitoring application in a distributed production environment. Strong reliability ensures each complex event is detected and delivered exactly once to each application entity, and cannot be guaranteed by the naive application of established replication principles. We present a replication scheme which ensures strong reliability in an asynchronous system model and can be applied to an arbitrary distributed CEP system. The algorithm tolerates f simultaneous failures by introducing f additional replicas for each correlation node. We prove correctness as well as evaluate the overhead introduced by the algorithm. Results show, that the overhead scales linearly with the number of deployed replicas and the node failure rate.

SUMMARY Current distributed publish/subscribe systems consider all participants to have similar QoS requirements and contribute equally to the system's resources. However, in many real‐world applications, the message delay tolerance of individual participants may differ widely. Disseminating messages according to individual delay requirements not only allows for the satisfaction of user‐specific needs, but also significantly improves the utilization of the resources that participants contribute to a publish/subscribe system. In this article, we propose a peer‐to‐peer‐based approach to satisfy the individual delay requirements of subscribers in the presence of bandwidth constraints. Our approach allows subscribers to dynamically adjust the granularity of their subscriptions according to their bandwidth constraints and delay requirements. Subscribers maintain the overlay in a decentralized manner, exclusively establishing connections that satisfy their individual delay requirements, and that provide messages exactly meeting their subscription granularity. The evaluations show that for many practical workloads, the proposed publish/subscribe system can scale up to a large number of subscribers and performs robustly in a very dynamic setting. Copyright © 2011 John Wiley & Sons, Ltd.

Real-time diagnostic simulations are one challenging application domain that is expected to introduce high requirements to global sensor applications. Besides having hard constraints on latency bounds at which data needs to be processed, such simulation applications will impose high requirements with respect to available bandwidth. Predictors, originally introduced in the domain of wireless sensor networks for energy saving, are one appealing solution to provide real-time estimates and at the same time significantly reduce the data rates. While in the setting of wireless sensor networks many prediction models have been analyzed, their behavior and use is unclear when applied to distributed data streams where aggregation results are typically processed over multilevel hierarchies. In the context of weather simulations, we propose a distributed R-Tree-based aggregation algorithm that allows for efficient reuse of aggregate queries. In the setting of real temperature readings taken from weather stations during one month, we study the trade-off between updates of the prediction model and the precision of the predicted values. Our evaluations indicate that even in situations where complex prediction models are expected to perform best, simple prediction models give higher benefits with respect to saving bandwidth while providing similar data accuracy.

Communication networks play a fundamental role in the response to a massive catastrophe, like an earthquake or a large-scale terrorist attack to a major urban area. In such situations, command centres must be able to rely on a fully operational communication network, for example to learn about on-going situations and allocate and guide the rescue teams. Communication is bidirectional: once in the field, these teams will feed the command centre with a more accurate view of the situation, contributing to the efficient allocation of the resources. Failures in this network, even if localised to some of the regions affected by the catastrophe, can have costs both monetary and in human lives. In this position paper, we propose the creation of a redundant, best-effort, emergency communication network that could serve to mitigate localised failures using off-the-shelf widespread technology. We give an overview of an architecture for a backup network, highlight the possible advantage of such an architecture to disaster management and discuss challenges that need to be overcome in realising it.

LYDIAN is an environment to support the teaching and learning of distributed algorithms. It provides a collection of distributed algorithms as well as continuous animations. Users can combine algorithms and animations with arbitrary network structures defining the interconnection and behavior of the distributed algorithm. Further, it facilitates the creation of algorithm descriptions as well as the creation of network structures. This makes LYDIAN a flexible tool to be used with students with different skills and backgrounds. This article gives an overview about various ideas and concepts behind LYDIAN by describing in detail the framework for an educational visualization and simulation environment for learning/teaching distributed algorithms as well as discussing possible extensions, which may improve possibilities for user interaction. Moreover, in our effort to understand better what visualization and simulation environments, such as LYDIAN, need to provide, we show results taken from a case study integrating LYDIAN in an undergraduate distributed-systems course.

Recent decentralised event-based systems have focused on providing event delivery which scales with increasing number of processes. While the main focus of research has been on ensuring that processes maintain only a small amount of information on maintaining membership and routing, an important factor in achieving scalability for event-based peer-to-peer dissemination system is the number of events disseminated at the same time. This work presents a dynamic and fault tolerant cluster management method which can be used to coordinate concurrent access to resources in a peer-to-peer system. In the context of event-based dissemination systems the cluster management can be used to control the number of concurrently disseminated events. We present and analyse an algorithm implementing the proposed cluster management model in a fault-tolerant and decentralised way. The algorithm provides for each cluster a limited set of tickets. A process which has obtained a ticket may send events corresponding to the resources of the cluster. The algorithm guarantees that no two processes ever issue an event corresponding to the same ticket at the same time. The cluster management model on its own has interesting properties which can be useful for many peer-to-peer applications.

Distributed algorithms can be difficult to understand as well as to teach. A way to provide students with an experience of the execution of a distributed algorithm is the use of a simulation-visualisation environment. In this work we present a case study of integrating a simulation-visualisation environment into a distributed system course. We evaluate a distributed system assignment in which students used LYDIAN, an extensible library for distributed algorithms and animations, to implement their algorithms. In our study neither the teachers nor the students had earlier class experience with LYDIAN. The feedback received gives valuable information on what simulation-visualisation environments for distributed algorithms need to provide in order to be successfully used in class. We are not aware of any similar study in the area of distributed computing. However, the feedback we have received shows the significance of such evaluations to help users improve their performance and help them to acknowledge the wealth of tools they are provided.

In multipeer communication decentralized probabilistic protocols have received a lot of attention because of their robustness against faults in the communication traffic and their potential to provide scalability for large groups. These protocols provide a probabilistic guarantee for a propagated event to reach every group member. Recent work aims to improve the scalability of such protocols by reducing memory requirements. In saving memory resources, the history buffer, which is used to "remember" received events and to prevent multiple deliveries of events to the application, plays a very significant role. We examine how the buffer size should be chosen to challenge the multiple delivery problems. Further, we propose and evaluate several methods of optimizing the dissemination of events in order to provide high reliability and reduce the number of multiple deliveries at the same time.

EnViDiA is an extensible environment that visualizes the execution of distributed algorithms by using the visualization enhancements offered by Virtual Reality technology. It addresses to represent the complex flow of information tied with the execution of a distributed algorithm in a way that also novices can easily develop a first understanding of the algorithm behavior. In difference to already existing tools it represents the communication structure in a 3D-model in which users are immersed. This way a natural interaction based on real world behavior is possible.The algorithm must work correctly using any arbitrary inter¿connection of processes represented by a communication graph. In contrast to ordinary 2D-worlds, complex non-planar graph models can be nicely represented in 3D with the perspective adapting to the movements of the user. Further, the orientation in the 3D-world is facilitated providing spatial sound. It assists the user becoming aware of the important system events.Students working within such an environment are more active since they walk or fly through the distributed system world in a game like scenario. Unlike the textbook approach students perceive a whole system execution instead of a series of snapshots for which students may experience difficulties in connecting them. The given experience is intended to help the students to follow better the formal descriptions and analysis of such algorithms.Undergraduate students have developed EnViDiA as part of the LYDIAN [2] project. The animation framework was designed for the Chalmers VR-Cube [1], an immersive VR environment and it is based on the problems the students experienced themselves when studying distributed algorithms for the first time.Although EnViDiA is intended to be used in an immersive VR environment, it is also possible to use EnViDiA in a simpler version on ordinary desktop computers supporting 3D-graphics (c.f. Figure 1). At its current state EnViDiA supports three distributed algorithms namely simple broadcast, broadcast with acknowledgement and resource allocation based on the algorithm by Ricart and Agrawala. The algorithms are taught in a basic distributed system course at Chalmers University of Technology.The development is about to be continued as part of the LYDIAN [2] project. Besides adding more algorithms and evaluating the tool at its current state, the main focus is on providing features to support multiple user collaboration, which are tested at the distributed concept of self-stabilization.

We describe and evaluate an experiment where actors were used to simulate the behaviour of processes in a distributed system in order to explain the concept of self-stabilisation in a graduate course on distributed systems.A self-stabilising system is one that ensures that the system's behaviour eventually stabilises to a safe subset of states regardless of the initial state. Protocols satisfying this elegant property, which enables a system to recover from transient failures that can alter the state of the system, are often hard to understand, especially for students that have not studied distributed computing and systems before.The experiment was part of an introductory course on distributed computing and systems for graduates in October 2000. The purpose of this interactive animation was to introduce to the students the basic concepts behind self-stabilisation (eligible states, transient faults, execution convergence) before their formal introduction.All of the students had a degree either in mathematics or computing science and had taken a course on algorithms before. However, most of the students did not have a background in distributed systems or distributed algorithms. The latter was not only the motivation for preparing this method of presentation but also what made this a challenging effort.The feedback from the class was that the concept and this teaching method were very well received. We could observe that their understanding evolved to the point that they were able to successfully come up with ideas for solutions and argue for/prove their correctness. As suggested in [1], dramatisation of executions can help the students to understand new issues and complications. This work shows that this is true even for graduate level courses. In our experiment we could conclude that dramatisation can be almost as powerful as a programming exercise in the teaching process; sometimes even more efficient, especially when we need to teach new concepts to an audience with diverse educational backgrounds. In analysing the results of our method we make a combination of the qualitative and quantitative approaches [4].

No abstract available.

We present our work on building interactive continuous visualisations of distributed algorithms for educational purposes. The animations are comprised by a set of visualisation windows. The visualisation windows are designed so that they demonstrate i) the different behaviours of the algorithms while running in different systems, ii) the different behaviours that the algorithms exhibit under different timing and workload of the system iii) the time and space complexities of the algorithms and iv) the "key ideas" of the functionality of the algorithms. Visualisations have been written for a set of lO algorithms that are tought in a Distributed Algorithms advanced undergraduate course.

No abstract available.