Dynamic data streams may contain concept drifts, which are data distribution changes that affect the underlying data model. Different types of concept drifts can occur within a real-world data stream. Majority of current study focuses on detecting one type of drift or detecting drift with labeled data, which is not always available in real-world scenarios. This study focuses on detecting all types of drift with limited labeled samples. Ensemble Framework for Drift Detection (EFDD) is proposed. The ensemble approach combines drift detection algorithms that can detect different types of drifts. Detection results from these algorithms are summarized using a novel voting mechanism called "voting by type". Experiments were carried out with one synthetic dataset and three real-world datasets. Experimental results show EFDD can achieve significant improvement with p <; 0.05 using z-score test when comparing to drift detection algorithms that detect only a few types of drift.

In order to obtain high quality labels, it is important to recognize and tackle noisy workers in crowdsourcing applications. In particular, spam workers, who randomly assign labels to items, can greatly degrade the crowdsourced label quality. As such, we propose a semi-supervised worker filtering (SWF) approach to filter this type of workers among the crowd. The SWF model recognizes spam workers by utilizing a limited set of gold truths. An optimization based truth discovery framework, which minimizes the total errors reside workers' labels, is integrated with the semi-supervised worker filtering approach (SWF-TD) to infer the true labels for unlabeled items. The efficacy of the proposed methodology is demonstrated on both synthetic and real-world datasets. The experimental analysis on real world datasets showed that by using around 40% gold truths as priori knowledge, it is possible that SWF-TD approach provides similar performance to the fully labeled worker filtering model.