The following work addresses the problem of frameworks for data stream processing that can be used to evaluate the solutions in an environment that resembles real-world applications. The definition of structured frameworks stems from the need to reliably assess data stream classification methods, considering the constraints of delayed label access, the costs of their acquisition and the costs of model adaptation. The current experimental evaluation often boundlessly exploits the assumption of the immediate label access to monitor the recognition quality and adapt the methods to the changing concepts. The problem is leveraged by reviewing currently described methods and techniques for data stream processing and verifying their outcomes in simulated environment . This work defines a taxonomy of data stream processing frameworks and presents four processing schemes that link the tasks of drift detection and classification while considering a natural phenomenon of label delay . The presented research shows that classification quality is significantly affected not only by the disruptive phenomenon of concept drifts and label delay , but also by the undertaken processing scheme that describes the flow of labels in the recognition system. Considering a specific processing framework depending on real-world constraints proves to be a critical aspect of reliable and realistic experimental evaluation.

The article presents the torchosr module – a Python package compatible with PyTorch library – offering functionality and models dedicated to Open Set Recognition in Deep Neural Networks. Included software offers two frequently used base recognition methods in the field and a set of functions for handling datasets, enabling the generation of derived datasets, where some classes are considered unknown and used only in the testing process. Code base is enhanced with a set of helper functions, facilitating model validation process. The main goal of the proposal is to simplify and promote the correct experimental evaluation, where experiments are carried out on a large number of derivative sets with various Openness, related to the cardinality of known and unknown classes, and class-to-category assignments. The authors hope that methods available in the package will become a source of a correct and open-source implementation of the relevant baseline and state-of-the-art solutions in the domain.

Abstract Concept drift in data stream processing remains an intriguing challenge and states a popular research topic. Methods that actively process data streams usually employ drift detectors, whose performance is often based on monitoring the variability of different stream properties. This publication provides an overview and analysis of metafeatures variability describing data streams with concept drifts. Five experiments conducted on synthetic, semi-synthetic, and real-world data streams examine the ability of over 160 metafeatures from 9 categories to recognize concepts in non-stationary data streams. The work reveals the distinctions in the considered sources of streams and specifies 17 metafeatures with a high ability of concept identification.

This publication presents the Complexity Drift Detector (C2D) – the method for detecting a concept shift in the data stream based on the classification task complexity measures. The method belongs to the group of detectors agnostic to the recognition quality of the base classifier. The possibility of selecting a set of difficulty measures taken into account during the data stream processing allows applying the method to many tasks in which the detection of a classification task complexity change is expected. The publication includes experiments analyzing the hyperparameters’ influence on the operation of the method and a broad comparative experiment comparing the proposed algorithm with state-of-the-art solutions. The experiments were carried out on synthetic data streams of different dimensions and with different concept drift characteristics, also presenting the effects of processing real-world data streams. The results of the conducted research confirm the high efficiency of the method in detecting concept changes, sensitive not only to the fact of drift occurrence but also to its dynamics.

Among the recently published works in the field of data stream analysis -both in the context of classification task and concept drift detection -the deficit of real-world data streams is a recurring problem.This article proposes a method for generating data streams with given parameters based on real-world static data.The method uses onedimensional interpolation to generate sudden or incremental concept drifts.The generated streams were subjected to an exemplary analysis in the concept drift detection task with a detector ensemble.The method can potentially contribute to the development of methods focused on data stream processing.

With the processing of data streams, come inevitable challenges, such as changes in the prior (class drift) and posterior (concept drift) probability distribution over the processing time. Both these phenomena have a negative impact on the quality of the classification. Heavily imbalanced problems, which are often typical for real-world applications, bring additional processing difficulties. Classifiers are often biased towards the majority class and have difficulty identifying instances of categories described with a lower number of objects. The following article proposes a Prior Probability Assisted Classifier (2PAC), a method aiming to improve the classification quality of heavily imbalanced data streams with dynamic changes by using the estimated prior probability value and the correction of the classifier's decision for batch predictions. Presented extensive computer experiments, supported by statistical analysis, show the ability to improve the classification quality using the proposed method.

Despite the fact that real-life data streams may often be characterized by the dynamic changes in the prior class probabilities, there is a scarcity of articles trying to clearly describe and classify this problem as well as suggest new methods dedicated to resolving this issue. The following paper aims to fill this gap by proposing a novel data stream taxonomy defined in the context of prior class probability and by introducing the Dynamic Statistical Concept Analysis (DSCA) - prior probability estimation algorithm. The proposed method was evaluated using computer experiments carried out on 100 synthetically generated data streams with various class imbalance characteristics. The obtained results, supported by statistical analysis, confirmed the usefulness of the proposed solution, especially in the case of discrete dynamically imbalanced data streams (DDIS).