This paper treats tracking as a foreground/background classification problem and proposes an online semi- supervised learning framework. Initialized with a small number of labeled samples, semi-supervised learning treats each new sample as unlabeled data. Classification of new data and updating of the classifier are achieved simultaneously in a co-training framework. The object is represented using independent features and an online support vector machine (SVM) is built for each feature. The predictions from different features are fused by combining the confidence map from each classifier using a classifier weighting method which creates a final classifier that performs better than any classifier based on a single feature. The semi-supervised learning approach then uses the output of the combined confidence map to generate new samples and update the SVMs online. With this approach, the tracker gains increasing knowledge of the object and background and continually improves itself over time. Compared to other discriminative trackers, the online semi-supervised learning approach improves each individual classifier using the information from other features, thus leading to a more robust tracker. Experiments show that this framework performs better than state-of-the-art tracking algorithms on challenging sequences.

Illumination changes cause object appearance to change drastically and many existing tracking algorithms lack the capability to handle this problem. The Earth mover's distance (EMD) is a similarity measure that is more robust against illumination changes. However, EMD is computationally expensive and we therefore propose the differential EMD (DEMD) algorithm which computes the derivative of the EMD with respect to the object location so that the EMD does not need to be computed for every location in the tracking window. The fast differential formula is derived based on the sensitivity analysis of the simplex method as applied to the EMD formula. To further reduce the computation, signatures, i.e., variable-size descriptions of distributions, are employed as an object representation. The new algorithm models local background scenes as well as foreground objects to handle scale changes in a principled way. Extensive quantitative evaluation of the proposed algorithm has been carried out using benchmark sequences and the improvement over the standard mean shift tracker is demonstrated.