Data stream learning is an emerging machine learning paradigm designed for environments where data arrive continuously and must be processed in real time. Unlike traditional batch learning, which assumes access to a fixed dataset, stream learning addresses the unique challenges of non-stationary distributions, bounded memory, and strict computational constraints. These challenges are increasingly relevant across domains such as IoT, finance, cybersecurity, and environmental monitoring, where timely and adaptive decision-making is essential. This tutorial introduces key concepts and techniques in data stream learning, blending foundational theory with practical demonstrations. It features CapyMOA, an open-source library that provides efficient algorithm implementations through a high-level Python API. We demonstrate the use of this tool through practical examples, with all source code available at https://github.com/adaptive-machine-learning/CapyMOA, and supporting tutorials and installation guides accessible at https://capymoa.org/.

Abstract Gradient boosting has been extensively studied in batch learning. Recently, its streaming adaptation, Streaming Gradient Boosted Trees (Sgbt), has surpassed existing state-of-the-art random subspace and random patches methods for streaming classification under various drift scenarios. However, its application in streaming regression remains unexplored. Vanilla Sgbt with squared loss exhibits high variance when applied to streaming regression problems. To address this, we utilize bagging streaming regressors in this work to create Streaming Gradient Boosted Regression (Sgbr). Bagging streaming regressors are employed in two ways: first, as base learners within the existing Sgbt framework, and second, as an ensemble method that aggregates multiple Sgbts. Our extensive experiments on 11 streaming regression datasets, encompassing multiple drift scenarios, demonstrate that the Sgb(Oza), a variant of the first Sgbr category, significantly outperforms current state-of-the-art streaming regression methods in terms of both predictive power and computational cost.

Abstract Gradient Boosting is a widely-used machine learning technique that has proven highly effective in batch learning. However, its effectiveness in stream learning contexts lags behind bagging-based ensemble methods, which currently dominate the field. One reason for this discrepancy is the challenge of adapting the booster to new concept following a concept drift. Resetting the entire booster can lead to significant performance degradation as it struggles to learn the new concept. Resetting only some parts of the booster can be more effective, but identifying which parts to reset is difficult, given that each boosting step builds on the previous prediction. To overcome these difficulties, we propose Streaming Gradient Boosted Trees (Sgbt), which is trained using weighted squared loss elicited in XGBoost. Sgbt exploits trees with a replacement strategy to detect and recover from drifts, thus enabling the ensemble to adapt without sacrificing the predictive performance. Our empirical evaluation of Sgbt on a range of streaming datasets with challenging drift scenarios demonstrates that it outperforms current state-of-the-art methods for evolving data streams.

Stream Learning (SL) attempts to learn from a data stream efficiently. A data stream learning algorithm should adapt to input data distribution shifts without sacrificing accuracy. These distribution shifts are known as ”concept drifts” in the literature. SL provides many supervised, semi-supervised, and unsupervised methods for detecting and adjusting to concept drift. On the other hand, Continual Learning (CL) attempts to preserve previous knowledge while performing well on the current concept when confronted with concept drift. In Online Continual Learning (OCL), this learning happens online. This survey explores the intersection of those two online learning paradigms to find synergies. We identify this intersection as Online Streaming Continual Learning (OSCL). The study starts with a gentle introduction to SL and then explores CL. Next, it explores OSCL from SL and OCL perspectives to point out new research trends and give directions for future research.

Neural networks have enjoyed tremendous success in many areas over the last decade. They are also receiving more and more attention in learning from data streams, which is inherently incremental. An incremental setting poses challenges for hyperparameter optimization, which is essential to obtain satisfactory network performance. To overcome this challenge, we introduce Continuously Adaptive Neural networks for Data streams (CAND). For every prediction, CAND chooses the current best network from a pool of candidates by continuously monitoring the performance of all candidate networks. The candidates are trained using different optimizers and hyperparameters. An experimental comparison against three state-of-the-art stream learning methods, over 17 benchmark streaming datasets con-firms the competitive performance of CAND, especially on high-dimensional data. We also investigate two orthogonal heuristics for accelerating Cand,which trade-off small amounts of accuracy for significant run-time gains. We observe that training on small mini-batches yields similar accuracy to single-instance fully incremental training, even on evolving data streams.