The classification service over a stream of data is becoming an important offering for cloud providers, but users may encounter obstacles in providing sensitive data due to privacy concerns. While Trusted Execution Environments (TEEs) are promising solutions for protecting private data, they remain vulnerable to side-channel attacks induced by data-dependent access patterns. We propose a Privacy-preserving Data Stream Training and Inference scheme, called EnclaveTree, that provides confidentiality for user's data and the target models against a compromised cloud service provider. We design a matrix-based training and inference procedure to train the Hoeffding Tree (HT) model and perform inference with the trained model inside the trusted area of TEEs, which provably prevent the exploitation of access-pattern-based attacks. The performance evaluation shows that EnclaveTree is practical for processing the data streams with small or medium number of features. When there are less than 63 binary features,EnclaveTree is up to ~10x and ~9 faster than naïve oblivious solution on training and inference, respectively.

Pollution from wood burners has profound health implications for the general population. Typically, monitoring the level of airborne particulate matter, PM2.5, in these areas often requires making inferences about missing or corrupted readings. Air Quality inference in these cases often poses critical challenges. The factors can evolve over time, changing the distribution of data. Such changes in the distribution of data are known as concept drift. Moreover, air pollution inference for a location typically would require historical data to be collected for the location. We investigate five air quality studies in New Zealand rural towns. We explore two different research problems: (1) an adaptive recurrent drift algorithm to model recurrence patterns in PM2.5levels for a town with the ability to recover after accuracy deterioration after a concept drift using an adaptive recurrent drift algorithm, and (2) transfer learning for the data stream whereby we reuse a pre-trained air pollution inference model from a town as the starting point for an air pollution inference model on another town. We further investigate the relationship between the changes we detected and changes within the prediction horizon. We showed that the average accuracy of the air quality inference for the five towns is between 70% and 94% using the recurrent drift algorithm. We also show that transfer learning was advantageous between two of the five towns.

In the online data stream environment, a model is typically not effective until a sufficient number of data instances have been seen. One solution to alleviate this issue is using model transfer. A major gap in current research is that they do not address the cost-effectiveness of model transfer in the online context, where processing time is crucial as data instances continuously arrive at high-speed. Model transfer in data streams involves continuously adapting the model to new data in the target stream. Suppose the target stream contains hard-to-learn patterns not covered by the transferred model. In that case, adaptation may incur extra processing time with little accuracy performance gains compared to simply building a new model for the target stream from scratch. Therefore, considering the cost of model transfer and target model construction is essential to balance the accuracy performance gains against computation cost for cost-effective model transfer. To address this gap, we propose a framework called OPERA (Online Transfer using Phantom Tree for Real-Time Adaptation), that orchestrates transfer learning based on the accuracy gain and runtime tradeoffs between transferring and adapting a source model, versus no transfer and constructing a new model for the target stream. We conduct extensive empirical studies to show that our framework can balance accuracy performance gains and runtime for cost-effective transfer learning in the data stream environment.

In many real-world applications, data are often produced in the form of streams. Consider, for example, data produced by sensors. In data streams there can be concept drift where the distribution of the data changes. When we deal with multiple streams from the same domain, concepts that have occurred in one stream may occur in another. Therefore, being able to reuse knowledge across multiple streams can help models recover from concept drifts more quickly. A major challenge is that these data streams may be only partially identical and a direct adoption would not suffice. In this paper, we propose a novel framework to transfer both identical and partially identical concepts across different streams. In particular, we propose a new technique called Adaptive Online TrAdaBoost that tunes weight adjustments during boosting based on model performance. The experiments on synthetic data verify the desired properties of the proposed method, and the experiments on real-world data show the better performance of the method for data stream mining compared with its baselines.

Most machine learning algorithms rely on a set of hyperparameters, which are tuned either by domain experts or using automated methods. In the context of data streams, the complexity of this task is compounded by the need for tuning every time a concept drift occurs. The problem of automated tuning may lead to classification performance improvements. Current methods optimise the hyperparameters from scratch every time concept drift is detected. Additionally, meta-learning techniques developed for static machine learning tasks allow pre-built knowledge to adapt faster to new tasks. We propose a meta-learning approach for automated tuning in evolving data streams, specifically for tuning parameters in Adaptive Random Forest (ARF) and parameters for several drift detectors. We compare our approach against non-adaptive methods on both synthetic and real-world datasets. Empirical results show that our approach makes a compromises between predictive performance and resource consumption for ARF, and between true positives and false positives for drift detectors.

Concept drift detection is used to signal to a learning algorithm that there has been a change in the underlying distribution of the data stream. However, there is a delay in detecting the actual drifts, leading to performance loss between the start of the drift and the detection point. There are two major challenges in reducing such performance loss, specifically the difficulty in anticipating the location of the next drift point and determining the exact concept that will appear for timely concept adaptation. In this research, we leverage concept recurrences in data streams. We proposed a framework called Nacre, which can perform proactive drift detection and online updates to allow for smooth adaptation of concept drifts. We present a novel technique, called drift coordinator, that anticipates the next drift point and assesses the incoming concept. This will ultimately increase accuracy in the classification performance. We demonstrate that our method is able to learn and predict drift trends in streams with recurring drifts. This allows the anticipation of future changes which enables users and detection methods to be more proactive. We empirically show that our technique outperforms baselines in terms of accuracy, kappa, accuracy gain per drift and cumulative accuracy gain on both synthetic and real-world datasets.

Random forest is an ensemble method used to improve the performance of single tree classifiers. In evolving data streams, the classifier needs to be adaptive and work under constraints of space and time. One benefit of random forest is its ability to be executed in parallel. In our research we introduce a random forest model utilizing a hybrid of both GPU and CPU, called GPU-based State-Adaptive Random Forest (GSARF). We address the pre-existing challenges of adapting random forest for data streams, specifically in the area of continual learning. Our novel approach reuses previously seen trees in the random forest when previous concepts reappear. This allows us to retain prior knowledge and provide a more stable predictive accuracy when changes occur in the data stream. Our random forest for data streams stores three types of trees, foreground trees which are trees that are currently used in prediction, background trees which are trees that are built when we are aware of possible changes in the data streams, and candidate trees which are trees that had been highly used in the previous concepts, but are now discarded due to changes in the data stream. We store candidate trees as they may be potentially useful at a later period in a repository and can be accessed when needed. We empirically show our technique performs up to 138 times the speed compared to current CPU-based random forest benchmarks. Our approach has shown to outperform a baseline GPU-based approach in terms of cumulative accuracy performance.