Our demonstration introduces a novel system architecture which massively facilitates optimization in data stream management systems (DSMS). The basic idea is to decouple optimization from the operative system by means of a secondary optimization system, which bears the burden of determining new query plans. Within the secondary system, which typically runs on a separate machine, we utilize suitable statistical models of the original data streams to simulate them. As the simulation can run at much faster rates, we are able to examine and assess new query plans in a shorter period of time without running the risk of deteriorating the original plan; we only migrate practically approved plans into the operative system. In our demonstration, we will present our prototypical implementation of this optimization architecture. We will demonstrate the interaction between primary and secondary system as well as the key features of the whole optimization process.

A variety of real-world applications heavily relies on an adequate analysis of transient data streams. Due to the rigid processing requirements of data streams, common analysis techniques as known from data mining are not directly applicable. A fundamental building block of many data mining and analysis approaches is density estimation. It provides a well-defined estimation of a continuous data distribution, a fact, which makes its adaptation to data streams desirable. A convenient method for density estimation utilizes kernels. The computational complexity of kernel density estimation, however, renders its application to data streams impossible. In this paper, we tackle this problem and propose our Cluster Kernel approach which provides continuously computed kernel density estimators over streaming data. Not only do Cluster Kernels meet the rigid processing requirements of data streams, they also allocate only a constant amount of memory, even with the opportunity to adapt it dynamically to changing system resources. For this purpose, we develop an intelligent merge scheme for Cluster Kernels and utilize continuously collected local statistics to resample already processed data. We focus on Cluster Kernels for one-dimensional data streams, but also address the multi-dimensional case. We validate the efficacy of Cluster Kernels for a variety of real-world data streams in an extensive experimental study.

A variety of scientific and commercial applications requires an immediate analysis of transient data streams. Many approaches for analyzing data share the property that an estimation of the underlying data distribution is used as a fundamental building block. To estimate the density of a continuous data distribution, wavelet density estimation, a technique from the area of nonparametric statistics, is very appealing as it is theoretically well-founded and practically approved. For that reason, its application to data streams is highly promising; it provides a convenient way to analyze the characteristics of a stream. However, the heavy computational cost of wavelet density estimators renders their direct application to the streaming scenario impossible. In this work, we tackle this problem and present a novel approach to adaptive wavelet density estimators over data streams. Not only do our estimators meet the rigid processing requirements for data streams, they also adapt to changing system resources in a well-defined manner. A thorough experimental evaluation demonstrates the efficacy of our wavelet density estimators and shows their superiority to competing kernel- and histogram-based estimators.

A fundamental building block of many data mining and analysis approaches is density estimation as it provides a comprehensive statistical model of a data distribution. For that reason, its application to transient data streams is highly desirable. A convenient, nonparametric method for density estimation utilizes kernels. However, its computational complexity collides with the rigid processing requirements of data streams. In this work, we present a new approach to this problem that combines linear processing cost with a constant amount of allocated memory. Our approach also supports a dynamic memory adaptation to changing system resources.

In order to support continuous queries over data streams, a plethora of suitable techniques as well as prototypes have been developed and evaluated in recent years. In particular, it is of utmost importance to confirm their necessity and feasibility in real-world applications. For that reason, we have successfully coupled our infrastructure for data stream processing (PIPES) with an industrial Production-to-Business software (i-Plant) dedicated to highly automated manufacturing processes.

A variety of real-world applications requires a meaningful online analysis of transient data streams. An important building block of many analysis tasks is the characterization of the underlying data distribution. Sophisticated techniques from the area of nonparametric statistics provide a well-defined estimation of continuous data distributions. The analysis of data streams may gain advantage of these techniques, however, the rigid processing requirements of streams render a direct application impossible. In our work, we tackle the adaptation of nonparametric techniques to streaming data. We concentrate on density estimation as it provides a convenient basis for the exploration of an unknown continuous data distribution. Specifically, we have developed kerneland wavelet-based density estimators for data streams in compliance with their processing requirements. Both techniques are incorporated into PIPES, our Java library for advanced data stream processing and analysis. In the demonstration, we present our nonparametric density estimators over data streams and show their performance for a variety of heterogeneous data streams from different real-world application scenarios. We also present the implementation of further analysis tasks on top of our estimators by means of illustrative use cases