Recently, multi-task correlation filters has drawn much attention in the object tracking field, which utilizes the multi-task learning (MTL) approach to explore the interdependencies among deep features for object tracking. However, the existing multi-task correlation filters based method fails to consider the relations between the correlation filters. To address this problem, a novel correlation filters based visual tracking method is proposed in this paper, with the integration of multi-task convolution operators and object detection. In our method, convolution and correction filters are jointly learnt through using the MTL technique, with the purpose of exploring not only the interdependencies of deep features but also the internal relevance of the convolution filters. In addition, object detection is introduced into our algorithm to handle the problem of object missing to ensure a better performance of our tracking method. Experiments on five benchmark datasets demonstrate that the proposed visual tracking method outperforms existing state-of-the-art approaches.

Recently, the correlation filter (CF) and Siamese network have become the two most popular frameworks in object tracking. Existing CF trackers, however, are limited by feature learning and context usage, making them sensitive to boundary effects. In contrast, Siamese trackers can easily suffer from the interference of semantic distractors. To address the above problems, we propose an end-to-end target-insight correlation network (TICNet) for object tracking, which aims at breaking the above limitations on top of a unified network. TICNet is an asymmetric dual-branch network involving a target-background awareness model (TBAM), a spatial-channel attention network (SCAN), and a distractor-aware filter (DAF) for end-to-end learning. Specifically, TBAM aims to distinguish a target from the background in the pixel level, yielding a target likelihood map based on color statistics to mine distractors for DAF learning. SCAN consists of a basic convolutional network, a channel-attention network, and a spatial-attention network, aiming to generate attentive weights to enhance the representation learning of the tracker. Especially, we formulate a differentiable DAF and employ it as a learnable layer in the network, thus helping suppress distracting regions in the background. During testing, DAF, together with TBAM, yields a response map for the final target estimation. Extensive experiments on seven benchmarks demonstrate that TICNet outperforms the state-of-the-art methods while running at real-time speed.

Recently, video-based person re-identification (re-ID) has drawn increasing attention in compute vision community because of its practical application prospects. Due to the inaccurate person detections and pose changes, pedestrian misalignment significantly increases the difficulty of feature extraction and matching. To address this problem, in this paper, we propose a Reference-Aided Part-Aligned (RAPA) framework to disentangle robust features of different parts. Firstly, in order to obtain better references between different videos, a pose-based reference feature learning module is introduced. Secondly, an effective relation-based part feature disentangling module is explored to align frames within each video. By means of using both modules, the informative parts of pedestrian in videos are well aligned and more discriminative feature representation is generated. Comprehensive experiments on three widely-used benchmarks, i.e. iLIDS-VID, PRID-2011 and MARS datasets verify the effectiveness of the proposed framework. Our code will be made publicly available.

Visual tracking is one of the fundamental tasks in computer vision with many challenges, and it is mainly due to the changes in the target's appearance in temporal and spatial domains. Recently, numerous trackers model the appearance of the targets in the spatial domain well by utilizing deep convolutional features. However, most of these CNN-based trackers only take the appearance variations between two consecutive frames in a video sequence into consideration. Besides, some trackers model the appearance of the targets in the long term by applying RNN, but the decay of the target's features degrades the tracking performance. In this article, we propose the antidecay long short-term memory (AD-LSTM) for the Siamese tracking. Especially, we extend the architecture of the standard LSTM in two aspects for the visual tracking task. First, we replace all of the fully connected layers with convolutional layers to extract the features with spatial structure. Second, we improve the architecture of the cell unit. In this way, the information of the target appearance can flow through the AD-LSTM without decay as long as possible in the temporal domain. Meanwhile, since there is no ground truth for the feature maps generated by the AD-LSTM, we propose an adversarial learning algorithm to optimize the AD-LSTM. With the help of adversarial learning, the Siamese network can generate the response maps more accurately, and the AD-LSTM can generate the feature maps of the target more robustly. The experimental results show that our tracker performs favorably against the state-of-the-art trackers on six challenging benchmarks: OTB-100, TC-128, VOT2016, VOT2017, GOT-10k, and TrackingNet.

Correlation filters combined with deep features have delivered impressive results in visual tracking task. However, existing approaches treat deep features produced by different network layers independently, limiting their representation power. To address this issue, this paper proposes a multi-task deep dual correlation filters (MDDCF) based method for robust visual tracking. First, a new multi-task learning scheme is designed to take full advantage of the multi-level features of deep networks, where target representation with individual features is regarded as a single task. As such, the interdependencies between different levels of features can be better explored. Second, we reformulate the objective function of the dual correlation filters and propose a new alternating optimization method, allowing joint training of the correlation filters and network parameters. Third, we design an effective object template update scheme which can well capture the target appearance variations. Extensive experimental evaluations on seven benchmark datasets show that the proposed MDDCF tracker performs favorably against state-ofthe-art methods.

A number of visual tracking methods achieve the state-of-the-art performance based on deep learning recently. However, most of these trackers utilize the deep neural network in regression task or classification task separately. In this paper, we propose an adversarial deep tracking framework. The framework is composed of a fully convolutional Siamese neural network (regression network) and a discriminative classification network. Then, we jointly optimize the regression network and the classification network by adversarial learning. In the uniform framework, the regression network and classification network can be trained end-to-end as a whole using large amounts of video training data sets. During the testing phase, the regression network generates a response map which reflects the location and the size of the target within each candidate search patch, and the classification network discriminates which response map is the best in terms of the corresponding template patch and candidate search patch. In addition, we propose an attention visualization algorithm for our tracker, and it reflects the area that attracts the attention of our tracker during tracking. The experimental results on three large-scale visual tracking benchmarks (OTB-100, TC-128, and VOT2016) demonstrate the effectiveness of the proposed tracking algorithm and show that our tracker performs comparably against the state-of-the-art trackers.

Feature extraction and representation is one of the most important components for fast, accurate, and robust visual tracking. Very deep convolutional neural networks (CNNs) provide effective tools for feature extraction with good generalization ability. However, extracting features using very deep CNN models needs high performance hardware due to its large computation complexity, which prohibits its extensions in real-time applications. To alleviate this problem, we aim at obtaining small and fast-to-execute shallow models based on model compression for visual tracking. Specifically, we propose a small feature distilled network (FDN) for tracking by imitating the intermediate representations of a much deeper network. The FDN extracts rich visual features with higher speed than the original deeper network. To further speed-up, we introduce a shift-and-stitch method to reduce the arithmetic operations, while preserving the spatial resolution of the distilled feature maps unchanged. Finally, a scale adaptive discriminative correlation filter is learned on the distilled feature for visual tracking to handle scale variation of the target. Comprehensive experimental results on object tracking benchmark datasets show that the proposed approach achieves 5× speed-up with competitive performance to the state-of-the-art deep trackers.

Correlation filters (CFs) have been extensively used in tracking tasks due to their high efficiency although most of them regard the tracked target as a whole and are minimally effective in handling partial occlusion. In this study, we incorporate a part-based strategy into the framework of CFs and propose a novel multipart correlation tracker with triangle-structure constraints. Specifically, we train multiple CFs for the global object and local parts, which are then jointly applied to obtain the correlation response of any candidate during tracking. The tracker is robust in handling partial occlusion because of the use of part-based representation. The remaining global representation can contribute reliable cues in cases wherein several local filters drift away in a specific scene. We further propose a triangle-structure model to measure the structural similarity of candidates. The model employs multiple triangles to determine the spatial relationship among parts and helps constrain the location of the target. Moreover, we introduce an effective part selection scheme based on energy and integrity, which is generally applicable to part-tracking models. Extensive experiments on two public benchmarks demonstrate the superiority of the proposed method over the state-of-the-art approaches.

Correlation filter has been demonstrated remarkable success for visual tracking recently. However, most existing methods often face model drift caused by several factors, such as unlimited boundary effect, heavy occlusion, fast motion, and distracter perturbation. To address the issue, this paper proposes a unified dynamic collaborative tracking framework that can perform more flexible and robust position prediction. Specifically, the framework learns the object appearance model by jointly training the objective function with three components: target regression submodule, distracter suppression submodule, and maximum margin relation submodule. The first submodule mainly takes advantage of the circulant structure of training samples to obtain the distinguishing ability between the target and its surrounding background. The second submodule optimizes the label response of the possible distracting region close to zero for reducing the peak value of the confidence map in the distracting region. Inspired by the structure output support vector machines, the third submodule is introduced to utilize the differences between target appearance representation and distracter appearance representation in the discriminative mapping space for alleviating the disturbance of the most possible hard negative samples. In addition, a CUR filter as an assistant detector is embedded to provide effective object candidates for alleviating the model drift problem. Comprehensive experimental results show that the proposed approach achieves the state-of-the-art performance in several public benchmark data sets.

Gesture is a natural interface in interacting with wearable devices such as VR/AR helmet and glasses. The main challenge of gesture recognition in egocentric vision arises from the global camera motion caused by the spontaneous head movement of the device wearer. In this paper, we address the problem by a novel recurrent 3D convolutional neural network for end-to-end learning. We specially design a spatiotemporal transformer module with recurrent connections between neighboring time slices which can actively transform a 3D feature map into a canonical view in both spatial and temporal dimensions. To validate our method, we introduce a new dataset with sufficient size, variation and reality, which contains 83 gestures designed for interaction with wearable devices, and more than 24,000 RGB-D gesture samples from 50 subjects captured in 6 scenes. On this dataset, we show that the proposed network outperforms competing state-of-the-art algorithms. Moreover, our method can achieve state-of-the-art performance on the challenging GTEA egocentric action dataset.

The goal of object tracking is to estimate the state and trajectory of an interested target in a video sequence, thus both spatial and temporal information are of critical importance for tracking. However, most existing trackers usually determine targets just by the judgement like confidence from a single frame, which tend to treat tracking as a static detecting problem while neglecting the spatial-temporal relationship. In this paper, we propose a novel tracking method of online trajectory optimization with multi-feature fusion (TOFF). Considering the trajectory continuity, the accurate targets are determined by estimating the optimal short trajectories over all the video fragments, which can be obtained by the observation models that are iteratively updated based on the selected most reliable proposals. By employing the structured samples instead of binary-labeled samples, we construct a structured output model with representing descriptor of multi-feature fusion as the basic tracker. Extensive experiments on various challenging image sequences demonstrate the superiority of our method to several state-of-the-art methods.

How to track an arbitrary object in video is one of the main challenges in computer vision, and it has been studied for decades. Based on hand-crafted features, traditional trackers show poor discriminability for complex changes of object appearance. Recently, some trackers based on convolutional neural network (CNN) have shown some promising results by exploiting the rich convolutional features. In this paper, we propose a novel DenseTracker based on a mutli-task dense convolutional network. To learn a more compact and discriminative representation, we adopt a dense block structure to ensemble features from different layers. Then a multitask loss is designed to accurately predict the object position and scale by joint learning of box regression and pair-wise similarity. Furtherly, the DenseTracker is trained end-to-end on large-scale datasets including ImageNet Video (VID) and ALOV300++. The DenseTracker runs in 25 fps on GPU and achieves the state-of-the-art performance on two public benchmarks of OTB50 and VOT2016.

Recently, the compressive tracking (CT) method has attracted much attention due to its high efficiency, but it cannot well deal with the large scale target appearance variations due to its data-independent random projection matrix that results in less discriminative features. To address this issue, in this paper, we propose an adaptive CT approach, which selects the most discriminative features to design an effective appearance model. Our method significantly improves CT in three aspects. First, the most discriminative features are selected via an online vector boosting method. Second, the object representation is updated in an effective online manner, which preserves the stable features while filtering out the noisy ones. Furthermore, a simple and effective trajectory rectification approach is adopted that can make the estimated location more accurate. Finally, a multiple scale adaptation mechanism is explored to estimate object size, which helps to relieve interference from background information. Extensive experiments on the CVPR2013 tracking benchmark and the VOT2014 challenges demonstrate the superior performance of our method.

Designing effective feature and handling the model drift problem are two important aspects for online visual tracking. For feature representation, gradient and color features are most widely used, but how to effectively combine them for visual tracking is still an open problem. In this paper, we propose a rich feature descriptor, MC-HOG, by leveraging rich gradient information across multiple color channels or spaces. Then MC-HOG features are embedded into the correlation tracking framework to estimate the state of the target. For handling the model drift problem caused by occlusion or distracter, we propose saliency proposals as prior information to provide candidates and reduce background interference. In addition to saliency proposals, a ranking strategy is proposed to determine the importance of these proposals by exploiting the learnt appearance filter, historical preserved object samples and the distracting proposals. In this way, the proposed approach could effectively explore the color-gradient characteristics and alleviate the model drift problem. Extensive evaluations performed on the benchmark dataset show the superiority of the proposed method.

Numerous approaches on object tracking have been proposed during the past decade with demonstrated success. However, most tracking algorithms are evaluated on limited video sequences and annotations. For thorough performance evaluation, we propose a large-scale database which contains 365 challenging image sequences of pedestrians and rigid objects. The database covers 12 kinds of objects, and most of the sequences are captured from moving cameras. Each sequence is annotated with target location and occlusion level for evaluation. A thorough experimental evaluation of 20 state-of-the-art tracking algorithms is presented with detailed analysis using different metrics. The database is publicly available and evaluation can be carried out online for fair assessments of visual tracking algorithms.

The sparse representation (SR)-based tracking framework generally considers the testing candidates and dictionary atoms individually, thus failing to model the structured information within data. In this paper, we present a robust tracking framework by exploiting the dual group structure of both candidate samples and dictionary templates, and formulate the SR at group level. The similar samples are encoded simultaneously by a few atom groups, which induces the inter-group sparsity, and also each group enjoys different internal sparsity. In this way, not only the potential commonality shared by the related candidates is taken into account but also the individual differences between samples are reflected. Then, we provide two effective optimization methods to solve our formulation by block-coordinate gradient descent and alternating direction method of multipliers, respectively, and make a comparison between them in terms of both effectiveness and efficiency. Finally, we embed the dual group structure model into the particle filter framework for visual tracking. Extensive experimental results demonstrate that our tracker achieves favorable performance against the state-of-the-art tracking methods.

Deep networks have been successfully applied to visual tracking by learning a generic representation offline from numerous training images. However, the offline training is time-consuming and the learned generic representation may be less discriminative for tracking specific objects. In this paper, we present that, even without offline training with a large amount of auxiliary data, simple two-layer convolutional networks can be powerful enough to learn robust representations for visual tracking. In the first frame, we extract a set of normalized patches from the target region as fixed filters, which integrate a series of adaptive contextual filters surrounding the target to define a set of feature maps in the subsequent frames. These maps measure similarities between each filter and useful local intensity patterns across the target, thereby encoding its local structural information. Furthermore, all the maps together form a global representation, via which the inner geometric layout of the target is also preserved. A simple soft shrinkage method that suppresses noisy values below an adaptive threshold is employed to de-noise the global representation. Our convolutional networks have a lightweight structure and perform favorably against several state-of-the-art methods on the recent tracking benchmark data set with 50 challenging videos.

Correlation filter based tracking has attracted many researchers’ attention in recent years for high efficiency and robustness. Most existing works focus on exploiting different characteristics with correlation filters for visual tracking, e.g. circulant structure, kernel trick, effective feature representation and context information. However, how to handle the scale variation and the model drift is still an open problem. In this paper, we propose a collaborative correlation tracker to deal with the above problems. Firstly, we extend the correlation tracking filter by embedding the scale factor into the kernelized matrix to handle the scale variation. Then a novel long-term CUR filter for detection is learnt efficiently with random sampling to alleviate model drift by detecting effective object candidates in the collaborative tracker. In this way, the proposed approach could estimate the object state accurately and handle the model drift problem effectively. Extensive experiments show the superiority of the proposed method.

Appearance-based human re-identification is challenging due to different camera characteristics, varying lighting conditions, pose variations across camera views, etc. Recent studies have revealed that color information plays a critical role on performance. However, two problems remain unclear: (1) how do different color descriptors perform under the same scene in re-identification problem? and (2) how can we combine these descriptors without losing their invariance property and distinctiveness power? In this paper, we propose a novel ensemble model that combines different color descriptors in the decision level through metric learning. Experiments show that the proposed system significantly outperforms state-of-the-art algorithms on two challenging datasets (VIPeR and PRID 450S). We have improved the Rank 1 recognition rate on VIPeR dataset by 8.7%.

Object tracking has been one of the most important and active research areas in the field of computer vision. A large number of tracking algorithms have been proposed in recent years with demonstrated success. However, the set of sequences used for evaluation is often not sufficient or is sometimes biased for certain types of algorithms. Many datasets do not have common ground-truth object positions or extents, and this makes comparisons among the reported quantitative results difficult. In addition, the initial conditions or parameters of the evaluated tracking algorithms are not the same, and thus, the quantitative results reported in literature are incomparable or sometimes contradictory. To address these issues, we carry out an extensive evaluation of the state-of-the-art online object-tracking algorithms with various evaluation criteria to understand how these methods perform within the same framework. In this work, we first construct a large dataset with ground-truth object positions and extents for tracking and introduce the sequence attributes for the performance analysis. Second, we integrate most of the publicly available trackers into one code library with uniform input and output formats to facilitate large-scale performance evaluation. Third, we extensively evaluate the performance of 31 algorithms on 100 sequences with different initialization settings. By analyzing the quantitative results, we identify effective approaches for robust tracking and provide potential future research directions in this field.

The past decade has witnessed significant progress in object detection and tracking in videos. In this paper, we present a model for collaboration between a pre-trained object detector and multiple single object trackers in the particle filter tracking framework. For each frame, we construct an association between the trackers and the detections, and when a tracker is successfully associated to a detection, we treat this detection as the key-sample for this tracker. We present a dual motion model that incorporates the associated detections with the object dynamics. Then, a likelihood function provides different weights for the propagated and the newly created particles, reducing the effect of false positives and missed detections in the tracking process. In addition, we use generative and discriminative appearance models to maximize the appearance variation among the targets. The performance of the proposed algorithm compares favorably with that of the state-of-the-art approaches on three public sequences.

In visual tracking, holistic and part-based representations are both popular choices to model target appearance. The former is known for great efficiency and convenience, while the latter for robustness against local appearance or shape variations. Based on nonnegative matrix factorization (NMF), we propose a novel visual tracker that takes advantage of both groups. The idea is to model the target appearance by a nonnegative combination of nonnegative components learned from examples observed in previous frames. To adjust NMF to the tracking context, we include sparsity and smoothness constraints in addition to the nonnegativity one. Furthermore, an online iterative learning algorithm, together with a proof of convergence, is proposed for efficient model updating. Putting these ingredients together with a particle filter framework, the proposed tracker, constrained online nonnegative matrix factorization (CONMF), achieves robustness to challenging appearance variations and nontrivial deformations while running in real time. We evaluate the proposed tracker on various benchmark sequences containing targets undergoing large variations in scale, pose, or illumination. The robustness and efficiency of CONMF is validated in comparison with several state-of-the-art trackers.

Object tracking is one of the most important components in numerous applications of computer vision. While much progress has been made in recent years with efforts on sharing code and datasets, it is of great importance to develop a library and benchmark to gauge the state of the art. After briefly reviewing recent advances of online object tracking, we carry out large scale experiments with various evaluation criteria to understand how these algorithms perform. The test image sequences are annotated with different attributes for performance evaluation and analysis. By analyzing quantitative results, we identify effective approaches for robust tracking and provide potential future research directions in this field.

Detecting and registering nonrigid surfaces are two important research problems for computer vision. Much work has been done with the assumption that there exists only one instance in the image. In this work, we propose an algorithm that detects and registers multiple nonrigid instances of given objects in a cluttered image. Specifically, after we use low level feature points to obtain the initial matches between templates and the input image, a novel high-order affinity graph is constructed to model the consistency of local topology. A hierarchical clustering approach is then used to locate the nonrigid surfaces. To remove the outliers in the cluster, we propose a deterministic annealing approach based on the Thin Plate Spline (TPS) model. The proposed method achieves high accuracy even when the number of outliers is nineteen times larger than the inliers. As the matches may appear sparsely in each instance, we propose a TPS based match growing approach to propagate the matches. Finally, an approach that fuses feature and appearance information is proposed to register each nonrigid surface. Extensive experiments and evaluations demonstrate that the proposed algorithm achieves promising results in detecting and registering multiple non-rigid surfaces in a cluttered scene.

Recently, sparse representation has been applied to visual tracking to find the target with the minimum reconstruction error from a target template subspace. Though effective, these L1 trackers require high computational costs due to numerous calculations for l1 minimization. In addition, the inherent occlusion insensitivity of the l1 minimization has not been fully characterized. In this paper, we propose an efficient L1 tracker, named bounded particle resampling (BPR)-L1 tracker, with a minimum error bound and occlusion detection. First, the minimum error bound is calculated from a linear least squares equation and serves as a guide for particle resampling in a particle filter (PF) framework. Most of the insignificant samples are removed before solving the computationally expensive l1 minimization in a two-step testing. The first step, named τ testing, compares the sample observation likelihood to an ordered set of thresholds to remove insignificant samples without loss of resampling precision. The second step, named max testing, identifies the largest sample probability relative to the target to further remove insignificant samples without altering the tracking result of the current frame. Though sacrificing minimal precision during resampling, max testing achieves significant speed up on top of τ testing. The BPR-L1 technique can also be beneficial to other trackers that have minimum error bounds in a PF framework, especially for trackers based on sparse representations. After the error-bound calculation, BPR-L1 performs occlusion detection by investigating the trivial coefficients in the l1 minimization. These coefficients, by design, contain rich information about image corruptions, including occlusion. Detected occlusions are then used to enhance the template updating. For evaluation, we conduct experiments on three video applications: biometrics (head movement, hand holding object, singers on stage), pedestrians (urban travel, hallway monitoring), and cars in traffic (wide area motion imagery, ground-mounted perspectives). The proposed BPR-L1 method demonstrates an excellent performance as compared with nine state-of-the-art trackers on eleven challenging benchmark sequences.

Recently sparse representation has been applied to visual tracker by modeling the target appearance using a sparse approximation over a template set, which leads to the so-called L1 trackers as it needs to solve an ℓ1norm related minimization problem for many times. While these L1 trackers showed impressive tracking accuracies, they are very computationally demanding and the speed bottleneck is the solver to ℓ1norm minimizations. This paper aims at developing an L1 tracker that not only runs in real time but also enjoys better robustness than other L1 trackers. In our proposed L1 tracker, a new ℓ1norm related minimization model is proposed to improve the tracking accuracy by adding an ℓ1norm regularization on the coefficients associated with the trivial templates. Moreover, based on the accelerated proximal gradient approach, a very fast numerical solver is developed to solve the resulting ℓ1norm related minimization problem with guaranteed quadratic convergence. The great running time efficiency and tracking accuracy of the proposed tracker is validated with a comprehensive evaluation involving eight challenging sequences and five alternative state-of-the-art trackers.

In this paper we study the problem of online aligning a newly arrived image to previously well-aligned images. Inspired by recent advances in batch image alignment using low rank decomposition [16], we treat the newly arrived image, after alignment, as being linearly and sparsely reconstructed by the well-aligned ones. The task is accomplished by a sequence of convex optimization that minimizes the l\-norm. After that, online basis updating is pursued in two different ways: (1) a two-stage incremental alignment for joint registration of a large image dataset which is known a prior, and (2) a greedy online alignment of dynamically increasing image sequences, such as in the tracking scenario. In (1), we first sequentially collect basis images that are easily aligned by checking their reconstruction residuals, followed by the second stage where all images are re-aligned one-by-one using the collected basis set. In (2), during the tracking process, we dynamically enrich the image basis set by the new target if it significantly distinguishes itself from existing basis images. While inheriting the benefits of sparsity, our method enjoys the great time efficiency and therefore be capable of dealing with large image set and real time tasks such as visual tracking. The efficacy of the proposed online robust alignment algorithm is verified with extensive experiments on image set alignment and visual tracking, in reference with state-of-the-art methods.

The recently proposed covariance region descriptor has been proven robust and versatile for a modest computational cost. The covariance matrix enables efficient fusion of different types of features, where the spatial and statistical properties, as well as their correlation, are characterized. The similarity between two covariance descriptors is measured on Riemannian manifolds. Based on the same metric but with a probabilistic framework, we propose a novel tracking approach on Riemannian manifolds with a novel incremental covariance tensor learning (ICTL). To address the appearance variations, ICTL incrementally learns a low-dimensional covariance tensor representation and efficiently adapts online to appearance changes of the target with only O(1) computational complexity, resulting in a real-time performance. The covariance-based representation and the ICTL are then combined with the particle filter framework to allow better handling of background clutter, as well as the temporary occlusions. We test the proposed probabilistic ICTL tracker on numerous benchmark sequences involving different types of challenges including occlusions and variations in illumination, scale, and pose. The proposed approach demonstrates excellent real-time performance, both qualitatively and quantitatively, in comparison with several previously proposed trackers.

Recently, sparse representation has been applied to visual tracking to find the target with the minimum reconstruction error from the target template subspace. Though effective, these L1 trackers require high computational costs due to numerous calculations for ℓ1minimization. In addition, the inherent occlusion insensitivity of the ℓ1minimization has not been fully utilized. In this paper, we propose an efficient L1 tracker with minimum error bound and occlusion detection which we call Bounded Particle Resampling (BPR)-L1 tracker. First, the minimum error bound is quickly calculated from a linear least squares equation, and serves as a guide for particle resampling in a particle filter framework. Without loss of precision during resampling, most insignificant samples are removed before solving the computationally expensive ℓ1minimization function. The BPR technique enables us to speed up the L1 tracker without sacrificing accuracy. Second, we perform occlusion detection by investigating the trivial coefficients in the ℓ1minimization. These coefficients, by design, contain rich information about image corruptions including occlusion. Detected occlusions enhance the template updates to effectively reduce the drifting problem. The proposed method shows good performance as compared with several state-of-the-art trackers on challenging benchmark sequences.

Visual tracking plays an important role in many computer vision tasks. A common assumption in previous methods is that the video frames are blur free. In reality, motion blurs are pervasive in the real videos. In this paper we present a novel BLUr-driven Tracker (BLUT) framework for tracking motion-blurred targets. BLUT actively uses the information from blurs without performing debluring. Specifically, we integrate the tracking problem with the motion-from-blur problem under a unified sparse approximation framework. We further use the motion information inferred by blurs to guide the sampling process in the particle filter based tracking. To evaluate our method, we have collected a large number of video sequences with significant motion blurs and compared BLUT with state-of-the-art trackers. Experimental results show that, while many previous methods are sensitive to motion blurs, BLUT can robustly and reliably track severely blurred targets.

Recently, the covariance region descriptor [1] has been proved robust and versatile for a modest computational cost. It enables efficient fusion of different types of features. Based on the covariance descriptor and the metric on Riemannian manifolds, we develop a robust Bayesian tracking framework via fragments-based representation in this paper. In this framework, the template object is represented by multiple image fragments or patches. Every patch votes on the possible state of the object in the current frame, by comparing its covariance descriptor with the corresponding image patch model. Tracking is then led by the Bayesian state inference framework in which a particle filter is used for propagating sample distributions over time. The weight of each particle is formulated by combining the votes of the patches using a robust statistic. Further, we extend the fast covariance computation to the Bayesian tracking problem, which makes the tracking procedure more efficient. We present extensive experimental results on challenging sequences, which demonstrate the robust tracking achieved by our algorithm.

Visual tracking is a challenging problem, as an object may change its appearance due to pose variations, illumination changes, and occlusions. Many algorithms have been proposed to update the target model using the large volume of available information during tracking, but at the cost of high computational complexity. To address this problem, we present a tracking approach that incrementally learns a low-dimensional covariance tensor representation, efficiently adapting online to appearance changes for each mode of the target with only ̃(1) computational complexity. Moreover, a weighting scheme is adopted to ensure less modeling power is expended fitting older observations. Both of these features contribute measurably to improving overall tracking performance. Tracking is then led by the Bayesian inference framework in which a particle filter is used to propagate sample distributions over time. With the help of integral images, our tracker achieves real-time performance. Extensive experiments demonstrate the effectiveness of the proposed tracking algorithm for the targets undergoing appearance variations.

In this paper, we propose a boosted interactively distributed particle filter (BIDPF) to address the problem of automatic multi-object tracking in the application of player tracking in broadcast soccer video. The interactively distributed particle filter technique (IDPF) is adopted to handle the mutual occlusions among targets. The proposal distribution using a mixture model that incorporates information from the dynamic model and the boosting detection is introduced into the IDPF framework. The boosting proposal distribution quickly detects targets, while the IDPF process keeps the identity of targets during mutual occlusions. Moreover, the foreground observation is extracted by using the color model of the playfield to speed up the boosting detection and reduce false alarms. The foreground is also used to develop a data-driven potential model to improve the IDPF performance. We test the proposed approach on several video sequences and the results demonstrate that our system is able to track a variable number of objects in a dynamic scene and correctly maintain their identities regardless of camera motion and frequent mutual occlusions.

The covariance region descriptor recently proposed in [1] has been proved robust and versatile for a modest computational cost. The covariance matrix enables efficient fusion of different types of features, where the spatial and statistical properties as well as their correlation are characterized. The similarity of two covariance descriptor is measured on Riemannian manifolds. Relying on the same metric, but within a probabilistic framework, we propose a novel tracking approach on Riemannian manifolds. The particle filtering technique allows us to better handle background clutter, as well as the temporary occlusions of the target. Furthermore, we extend the fast covariance computation to the tracking problem, which makes the tracking procedure more efficient. The proposed approach is robust to noises and much faster than the original search-based covariance tracker [2]. Extensive experimental results demonstrate greatly improved performance over classical color-based Bayesian tracker.

In this paper, a fast and robust multi-view lip localization algorithm in video is proposed. We consider lip localization as a binary classification problem, where a classifier is learned to distinguish between the lip and the region surrounding it. The classifier we use here is a histogram-based one which exploits the anthropometrical properties of the human face with the help of face scale normalization. Due to the perceptual uniformity and robustness for lip/skin color variations across different people, we adopt CIELUV color model to represent the color of lip. After classification, we propose a novel projection-cut algorithm by spatial deviation analysis (SDA) to locate the lip, which is effective to deal with the background clutters. Experimental results on teleplay videos demonstrate that the proposed approach is efficient and robust for lip localization.

In this paper, a real time and robust multi-view lip localization in video frames is proposed. The objective of this research is to exactly localize lip surrounding box with sub-pixel accuracy in low quality video frames. This is achieved by multi-view face detection in video frames, lip ROI extraction, color space transformation, vertical edge feature detection and lip localization by a feature-spatial profile analysis on a joint color-edge plane. One possible application of talking face detection in video is used to evaluate our work. Experimental results on TV programs and movies demonstrate that the proposed approach is efficient and robust in lip localization.