Continuously learning and serving from evolving streaming data and serving in real-time is a challenging problem. Traditionally, data is partitioned and processed in batches to train machine learning (ML) models. In industrial applications, static models’ performance drops over time (model degradation, concept drift), requiring new models to be trained with recent data and redeployed in production. The scientific community has been studying online and adaptive methods to address batch-learning limitations and continuously train AI tasks for industrial applications such as cyber-security, AIOps, anomaly scoring, and drift detection in stock markets. This paper deals with the MLOps aspects of deploying such online and dynamic models to address the requirements in the production systems for real-time applications. Our architectures - based on open-source tools such as Kafka and River - demonstrated how online learning methods could be scaled horizontally in production to meet the demands of a high-velocity streaming pipeline. We demonstrate an MLOps strategy to perform incremental learning from streaming data and continuously deploy the online learning model without pausing the inference pipeline. Indeed, the design satisfies requirements such as model versioning, monitoring, audibility and reproducibility of prediction in both a supervised and semi-supervised setting. Our experiments - for malicious URLs detection task - performed on high-dimensional and feature-evolving streaming data (more than 3 million features) establish the effectiveness and efficiency of online learning models compared to batch (static) machine learning regarding both time and space complexity. Finally, we provide some best practices on data engineering for deploying online models to process a real-time feature stream in production environments. Code is publicly available for reproducibility.

Knowledge Graphs (KG) are valuable information sources that store knowledge in a domain (healthcare, finance, e-commerce, cyber-security.). Most industrial KGs are dynamic by nature as they are updated regularly with streaming data (customer activity, network traffic, application logs, IT process). However, extracting insights from continuously updated data comes with major challenges, particularly in big data settings. In this paper, we address the following challenges: 1) ingesting heterogeneous data, 2) training and deployment of predictive models on continuously evolving data, and 3) implementation of data pipelines for updating and maintaining the KG in production. We cover multiple aspects of this process, from knowledge collection to its operationalization. We propose Stream2Graph, a stream-based system for building and updating the knowledge base dynamically in real time. Then we show how graph features can be used in downstream online machine learning models. The solution speeds up big data stream learning and knowledge extraction to enhance Graph-based AI applications. Experimental results show the effectiveness of our solution for knowledge base construction and improvement of big data learning capabilities. Using data from Stream2Graph resulted in speedups for training and inference time in the range from 547x to 2000x in downstream ML models. Finally, we provide the lessons learned from applying graph-based online learning on large-scale network processing high-velocity streaming data.

The growing need for predictive analytics over streaming data in the industry requires a flexible and continuously scalable big data system. In real-time big data applications (cybersecurity, AIOps, anomaly detection, predictive maintenance, IoT etc.), efficient machine learning models must be trained and industrialized within existing data processing plat-forms and industrial tools. This requires interoperability between various components: data collection, processing, summarization, modelling and analytics. Existing works focus on building AI models for big data, neglecting real-world challenges when integrating such models into an existing industrial production framework. In this paper, we propose StreamFlow, an operational data pipeline to address industrial challenges for continuous learning over big data streams. We also propose an online method using sliding windows to summarize high-velocity data. The final result of the framework is a feature vector that describes the underlying processes and is ready to use in machine learning tasks. Moreover, we showcase real-world applications such as automated feature engineering for real-time monitoring and online machine learning for event classification. The proposed system has been deployed within production in a banking system, processing billions of daily traffic operations. Our experiments demonstrate the effectiveness and performance of our approach by evaluating it at different levels: processing, summarization, improvement of machine learning performance and effectiveness in an industrial setting. In the case of downstream machine learning tasks, using summarized data generated by StreamFlow results in up to 2 orders of magnitude speedups in training time without compromising predictive performance.

Online clustering algorithms play a critical role in data science, especially with the advantages regarding time, memory usage and complexity, while maintaining a high performance compared to traditional clustering methods. This tutorial serves, first, as a survey on online machine learning and, in particular, data stream clustering methods. During this tutorial, state-of-the-art algorithms and the associated core research threads will be presented by identifying different categories based on distance, density grids and hidden statistical models. Clustering validity indices, an important part of the clustering process which are usually neglected or replaced with classification metrics, resulting in misleading interpretation of final results, will also be deeply investigated.

Streaming Machine Learning (SML) studies single-pass learning algorithms that update their models one data item at a time given an unbounded and often non-stationary flow of data (a.k.a., in presence of concept drift). Online class imbalance learning is a branch of SML that combines the challenges of both class imbalance and concept drift. In this paper, we investigate the binary classification problem of rebalancing an imbalanced stream of data in the presence of concept drift, accessing one sample at a time. We propose Continuous Synthetic Minority Oversampling Technique (C-SMOTE), a novel rebalancing meta-strategy to pipeline with SML classification algorithms. C-SMOTE is inspired by the popular SMOTE algorithm but operates continuously. We benchmark C-SMOTE pipelines on ten different groups of data streams. We bring empirical evidence that models learnt with C-SMOTE pipelines outperform models trained on imbalanced data stream without losing the ability to deal with concept drifts. Moreover, we show that they outperform other stream balancing techniques from the literature.

Data ubiquity highlights the need of efficient and adaptable data-driven solutions. In this paper, we present FAST AND SLOW LEARNING (FSL), a novel unified framework that sheds light on the symbiosis between batch and stream learning. FSL works by employing Fast (stream) and Slow (batch) Learners, emulating the mechanisms used by humans to make decisions. We showcase the applicability of FSL on the task of classification by introducing the FAST AND SLOW CLASSIFIER (FSC). A Fast Learner provides predictions on the spot, continuously updating its model and adapting to changes in the data. On the other hand, the Slow Learner provides predictions considering a wider spectrum of seen data, requiring more time and data to create complex models. Once that enough data has been collected, FSC trains the Slow Learner and starts tracking the performance of both learners. A drift detection mechanism triggers the creation of new Slow models when the current Slow model becomes obsolete. FSC selects between Fast and Slow Learners according to their performance on new incoming data. Test results on real and synthetic data show that FSC effectively drives the positive interaction of stream and batch models for learning from evolving data streams.

Detecting over-indebtedness, the difficulties meeting household payment commitments, poses multiple Big Data challenges for banking institutions. We present a novel data-driven framework for predicting over-indebtedness on realworld data. A warning mechanism that generates predictions 6 months ahead, improving the chances of financial recovery. This framework is based on the combination of feature selection and supervised learning techniques, and uses data balancing for fine-tuning the predictive models. We propose two versions of the framework based on state-of-the-art batch and streaming learning techniques. To the best of our knowledge, the proposed framework is the first to cast over-indebtedness prediction as a stream learning problem. The appeal of stream learning rises from the large amount of data continuously generated, and the fact that batch models become obsolete over time as financial data evolves, while stream models are continuously updated as new data is available. We use credit data from two banks from the Groupe BPCE (the second-largest banking institution in France) and apply multi-metric criteria to evaluate model performance and fairness. Test results show the framework's interbank applicability and that the proposed batch and stream frameworks outperform the current solution for both single and multi-metric criteria. Additionally, the generic structure of the framework serves as a template for systematically approaching similar classification problems.