Challenges caused by the acquisition condition of the images, the state of the objects, or the noise in the transmission of the images commonly exist in object detection. In those situations, the features of the objects extracted by CNNs contain certain uncertainty, which increases the difficulty of subsequent classification and regression. Towards enhancing the quality of the features, we propose a Feature Sampling Module (FSM), which learns multiple two-dimensional Gaussian distributions by the sampling network (SN) and applies those Gaussian masks to extract valid information of the features. With this sampling scheme, our method avoids learning the decision boundary from the low-quality features, making the overall model classification performance more robust. To ensure that the SN is capable of sampling the highest quality region, we design a novel sampling loss (SL) to measure the quality of the sampled features. Extensive experimental results validate the effectiveness of our proposed method.

Different from conventional face recognition, the gender discrepancy between parent and child is an inevitable issue for kinship verification. Father and daughter, or mother and son, may have different facial features due to gender differences, which renders kinship verification difficult. In view of this, this paper proposes a gender-invariant feature extraction and image-to-image translation network (Gender-FEIT) that learns a gender invariant face representation and produces the transgendered images simultaneously. In Gender-FEIT, the male (female) face is first projected to a feature representation through an encoder, then the representation is transformed into a female (male) face through the specific generator. A gender discriminator is imposed on the encoder, forcing to learn a gender invariant representation in an adversarial way. This representation preserves the high-level personal information of the input face but removes gender information, which is applicable to cross-gender kinship verification. Moreover, the competition between generators and image discriminators encourages to generate realistic-looking faces that can enlarge kinship datasets. This novel data augmentation method significantly improves the performance of kinship verification. Experimental results demonstrate the effectiveness of our method on two most widely used kinship databases.

Few-shot object detection (FSOD) aims at learning a detector that can fast adapt to previously unseen objects with scarce annotated examples. Existing methods solve this problem by performing subtasks of classification and localization utilizing a shared component in the detector, yet few of them take the distinct preferences towards feature embedding of two subtasks into consideration. In this paper, we carefully analyze the characteristics of FSOD, and present that a few-shot detector should consider the explicit decomposition of two subtasks, as well as leveraging information from both of them to enhance feature representations. To the end, we propose a simple yet effective Adaptive Fully-Dual Network (AFD-Net). Specifically, we extend Faster R-CNN by introducing Dual Query Encoder and Dual Attention Generator for separate feature extraction, and Dual Aggregator for separate model reweighting. In this way, separate state estimation is achieved by the R-CNN detector. Furthermore, we introduce Adaptive Fusion Mechanism to guide the design of encoders for efficient feature fusion in the specific subtask. Extensive experiments on PASCAL VOC and MS COCO show that our approach achieves state-of-the-art performance by a large margin, demonstrating its effectiveness and generalization ability.

Studies have shown that contextual information can promote the robustness of trackers. However, trackers based on convolutional neural networks (CNNs) only capture local features, which limits their performance. We propose a novel relevant context block (RCB), which employs graph convolutional networks to capture the relevant context. In particular, it selects the$k$largest contributors as nodes for each query position (unit) that contain meaningful and discriminative contextual information and updates the nodes by aggregating the differences between the query position and its contributors. This operation can be easily incorporated into the existing networks and can be easily end-to-end trained using a standard backpropagation algorithm. To verify the effectiveness of RCB, we apply it to two trackers, SiamFC and GlobalTrack, respectively, and the two improved trackers are referred to as Siam-RCB and GlobalTrack-RCB. Extensive experiments on OTB, VOT, UAV123, LaSOT, TrackingNet, OxUvA, and VOT2018LT show the superiority of our method. For example, our Siam-RCB outperforms SiamFC by a very large margin (up to 11.2% in the success score and 7.8% in the precision score) on the OTB-100 benchmark.

In visual tracking, usually only a small number of samples are labeled, and most existing deep learning based trackers ignore abundant unlabeled samples that could provide additional information for deep trackers to boost their tracking performance. An intuitive way to explain unlabeled data is to incorporate manifold regularization into the common classification loss functions, but the high computational cost may prohibit those deep trackers from practical applications. To overcome this issue, we propose a two-stage approach to a deep tracker that takes into account both labeled and unlabeled samples. The annotation of unlabeled samples is propagated from its labeled neighbors first by exploring the manifold space that these samples are assumed to lie in. Then, we refine it by training a deep convolutional neural network using both labeled and unlabeled data in a supervised manner. Online visual tracking is further carried out under the framework of particle filters with the presented manifold regularized deep model being updated every few frames. Experimental results on different tracking datasets demonstrate that our tracker outperforms most existing tracking approaches. The source code and results are available at: https://github.com/shenjianbing/MRCNNTracking.

In this paper, we propose a manifold regularized correlation tracking method with augmented samples. To make better use of the unlabeled data and the manifold structure of the sample space, a manifold regularization-based correlation filter is introduced, which aims to assign similar labels to neighbor samples. Meanwhile, the regression model is learned by exploiting the block-circulant structure of matrices resulting from the augmented translated samples over multiple base samples cropped from both target and nontarget regions. Thus, the final classifier in our method is trained with positive, negative, and unlabeled base samples, which is a semisupervised learning framework. A block optimization strategy is further introduced to learn a manifold regularization-based correlation filter for efficient online tracking. Experiments on two public tracking data sets demonstrate the superior performance of our tracker compared with the state-of-the-art tracking approaches.

We propose a method that obtains a discriminative visual dictionary and a nonlinear classifier for visual tracking tasks in a sparse coding manner based on the globally linear approximation for a nonlinear learning theory. Traditional discriminative tracking methods based on sparse representation learn a dictionary in an unsupervised way and then train a classifier, which may not generate both descriptive and discriminative models for targets by treating dictionary learning and classifier learning separately. In contrast, the proposed tracking approach can construct a dictionary that fully reflects the intrinsic manifold structure of visual data and introduces more discriminative ability in a unified learning framework. Finally, an iterative optimization approach, which computes the optimal dictionary, the associated sparse coding, and a classifier, is introduced. Experiments on two benchmarks show that our tracker achieves a better performance compared with some popular tracking algorithms.

Correlation filter-based tracking methods have accomplished competitive performance on accuracy and robustness, but there is still a huge potential in choosing suitable features. Recently, Convolutional Kernel Networks (CKN), which provide a fast and simple procedure to approximate kernel descriptors, have been proposed and achieved state-of-the-art performance in many vision tasks. In this paper, we present an adaptive tracker which integrates the kernel correlation filters with multiple effective CKN descriptors. By adopting a FlipFlop scheme, the weights of different features can be adjusted in the process of tracking to get better performance. Extensive experimental results on the OTB-2013 tracking benchmark show that our approach performs favorably against some representative state-of-the-art tracking algorithms.

In this paper, we present a novel part-based visual tracking method from the perspective of probability sampling. Specifically, we represent the target by a part space with two online learned probabilities to capture the structure of the target. The proposal distribution memorizes the historical performance of different parts, and it is used for the first round of part selection. The acceptance probability validates the specific tracking stability of each part in a frame, and it determines whether to accept its vote or to reject it. By doing this, we transform the complex online part selection problem into a probability learning one, which is easier to tackle. The observation model of each part is constructed by an improved supervised descent method and is learned in an incremental manner. Experimental results on two benchmarks demonstrate the competitive performance of our tracker against state-of-the-art methods.

As an important technique in multivariate statistical analysis, Canonical Correlation Analysis (CCA) has been widely used in face recognition. But existing CCA based face recognition methods need two kinds of expression for the same face sample, and usually suffers high computational complexity in dealing with large samples. In this paper, we present a supervised method called Randomized Canonical Correlation Discriminant Analysis (RCCDA) based on Randomized non-linear Canonical Correlation Analysis (RCCA) to make up for the shortage of CCA based face recognition methods. We first obtain basis vectors approximately with random features instead of the calculation of kernel matrix to improve the efficiency of computation, then we use these basis vectors to compute random optimal discriminant features which can reduce the dimension of face features while preserving as much discriminatory information as possible. The result of experiments on Extended Yale B, AR, ORL and FERET face databases demonstrates that the performance of our method compares favorably with some state-of-the-art algorithms.

Most existing discriminative tracking algorithms use a sampling-and-labeling strategy to collect examples and treat the training example collection as a task that is independent of classifier learning. However, the examples collected directly by sampling are neither necessarily informative nor intended to be useful for classifier learning. Updating the classifier with these examples might introduce ambiguity to the tracker. In this paper, we present a novel online discriminative tracking framework that explicitly couples the objectives of example collection and classifier learning. Our method uses Laplacian regularized least squares (LapRLS) to learn a robust classifier that can sufficiently exploit unlabeled data and preserve the local geometrical structure of the feature space. To ensure the high classification confidence of the classifier, we propose an active example selection approach to automatically select the most informative examples for LapRLS. Part of the selected examples that satisfy strict constraints are labeled to enhance the adaptivity of our tracker, which actually provides robust supervisory information to guide semisupervised learning. With active example selection, we are able to avoid the ambiguity introduced by an independent example collection strategy and to alleviate the drift problem caused by misaligned examples. Comparison with the state-of-the-art trackers on the comprehensive benchmark demonstrates that our tracking algorithm is more effective and accurate.

How to effectively organize local descriptors to build a global representation has a critical impact on the performance of vision tasks. Recently, local sparse representation has been successfully applied to visual tracking, owing to its discriminative nature and robustness against local noise and partial occlusions. Local sparse codes computed with a template actually form a three-order tensor according to their original layout, although most existing pooling operators convert the codes to a vector by concatenating or computing statistics on them. We argue that, compared to pooling vectors, the tensor form could deliver more intrinsic structural information for the target appearance, and can also avoid high dimensionality learning problems suffered in concatenation-based pooling methods. Therefore, in this paper, we propose to represent target templates and candidates directly with sparse coding tensors, and build the appearance model by incrementally learning on these tensors. We propose a discriminative framework to further improve robustness of our method against drifting and environmental noise. Experiments on a recent comprehensive benchmark indicate that our method performs better than state-of-the-art trackers.

Feature pooling in a majority of sparse coding-based tracking algorithms computes final feature vectors only by low-order statistics or extreme responses of sparse codes. The high-order statistics and the correlations between responses to different dictionary items are neglected. We present a more generalized feature pooling method for visual tracking by utilizing the probabilistic function to model the statistical distribution of sparse codes. Since immediate matching between two distributions usually requires high computational costs, we introduce the Fisher vector to derive a more compact and discriminative representation for sparse codes of the visual target. We encode target patches by local coordinate coding, utilize Gaussian mixture model to compute Fisher vectors, and finally train semi-supervised linear kernel classifiers for visual tracking. In order to handle the drifting problem during the tracking process, these classifiers are updated online with current tracking results. The experimental results on two challenging tracking benchmarks demonstrate that the proposed approach achieves a better performance than the state-of-the-art tracking algorithms.

Most existing tracking algorithms do not explicitly consider the motion blur contained in video sequences, which degrades their performance in real-world applications where motion blur often occurs. In this paper, we propose to solve the motion blur problem in visual tracking in a unified framework. Specifically, a joint blur state estimation and multi-task reverse sparse learning framework are presented, where the closed-form solution of blur kernel and sparse code matrix is obtained simultaneously. The reverse process considers the blurry candidates as dictionary elements, and sparsely represents blurred templates with the candidates. By utilizing the information contained in the sparse code matrix, an efficient likelihood model is further developed, which quickly excludes irrelevant candidates and narrows the particle scale down. Experimental results on the challenging benchmarks show that our method performs well against the state-of-the-art trackers.

Due to unavoidable appearance variations caused by occlusion, deformation, and other factors, classifiers for visual tracking are nonlinear as a necessity. Building on the theory of globally linear approximations to nonlinear functions, we introduce an elegant method that jointly learns a nonlinear classifier and a visual dictionary for tracking objects in a semi-supervised sparse coding fashion. This establishes an obvious distinction from conventional sparse coding based discriminative tracking algorithms that usually maintain two-stage learning strategies, i.e., learning a dictionary in an unsupervised way then followed by training a classifier. However, the treating dictionary learning and classifier training as separate stages may not produce both descriptive and discriminative models for objects. By contrast, our method is capable of constructing a dictionary that not only fully reflects the intrinsic manifold structure of the data, but also possesses discriminative power. This paper presents an optimization method to obtain such an optimal dictionary, associated sparse coding, and a classifier in an iterative process. Our experiments on a benchmark show our tracker attains outstanding performance compared with the state-of-the-art algorithms.

Recently, local sparse representation (LSR) has been successfully applied in visual tracking, owing to its discriminative nature and robustness against local noise and occlusions. It is note worthy that local sparse codes computed with a template form a 3-order tensor of their original layout, although most pooling operators convert it to a vector by concatenating or computing statistics on it. As compared to pooling vectors, tensor form could deliver more informative and structured representation for target appearance, and can also avoid high dimensionality learning problem suffered in concatenating pooling based methods. Motivated by above ideas, in this paper, we propose to represent target templates directly with sparse coding tensors, and build the appearance model by incrementally learning on these tensors. We further propose a discriminative framework to improve robustness against drifting and environment noise. Experiments on a recent comprehensive benchmark indicate that our method outperforms state-of-the-art trackers.

Kernel Principal Component Analysis (KPCA) is a popular feature extraction technique for face recognition. However, it often suffers from the high computational complexity problem, when dealing with large samples. Besides, KPCA is a holistic feature based approach, which means that it discards some useful discriminate local information. In this paper, we use Random Nonlinear Principal Component Analysis (RNPCA) and extract Local Ternary Patterns (LTP) features to improve them respectively. We calculate the kernel matrix by constructing random Fourier features, thus the computation efficiency is speeded up. The LTP features are also extracted, so the local texture information is preserved. In the classification section, we use distance metric learning to improve the classification ability of nearest neighbors classifier. Experimental results on AR, FERET, Yale, ORL face databases demonstrated the effectiveness of our method.

Sparse coding methods have achieved great success in visual tracking, and we present a strong classifier and structural local sparse descriptors for robust visual tracking. Since the summary features considering the sparse codes are sensitive to occlusion and other interfering factors, we extract local sparse descriptors from a fraction of all patches by performing a pooling operation. The collection of local sparse descriptors is combined into a boosting-based strong classifier for robust visual tracking using a discriminative appearance model. Furthermore, a structural reconstruction error based weight computation method is proposed to adjust the classification score of each candidate for more precise tracking results. To handle appearance changes during tracking, we present an occlusion-aware template update scheme. Comprehensive experimental comparisons with the state-of-the-art algorithms demonstrated the better performance of the proposed method.

In this paper, we propose to address online visual tracking on the basis of Local Coordinate Coding (LCC), which integrates the advantages of the discriminative method and the generative method. In the discriminative module, a nonlinear function is trained using the local coordinate codes of image patches to identify the foreground patches from background. In the generative module, we introduce a similarity function that takes the spatial structures of local patches in the target into account between the candidate and holistic templates by reconstruction error. To deal with appearance change during tracking, an online update method is introduced. The proposed tracking method is evaluated on different challenging video sequences with center location error, and experimental results demonstrate the good performance of our method.

Appearance modeling is a key issue for the success of a visual tracker. Sparse representation based appearance modeling has received an increasing amount of interest in recent years. However, most of existing work utilizes reconstruction errors to compute the observation likelihood under the generative framework, which may give poor performance, especially for significant appearance variations. In this paper, we advocate an approach to visual tracking that seeks an appropriate metric in the feature space of sparse codes and propose a metric learning based structural appearance model for more accurate matching of different appearances. This structural representation is acquired by performing multiscale max pooling on the weighted local sparse codes of image patches. An online multiple instance metric learning algorithm is proposed that learns a discriminative and adaptive metric, thereby better distinguishing the visual object of interest from the background. The multiple instance setting is able to alleviate the drift problem potentially caused by misaligned training examples. Tracking is then carried out within a Bayesian inference framework, in which the learned metric and the structure object representation are used to construct the observation model. Comprehensive experiments on challenging image sequences demonstrate qualitatively and quantitatively that the proposed algorithm outperforms the state-of-the-art methods.

When visual tracking is performed by human, we typically pay attention to some salient regions or points of the target instead of the whole target. Inspired by this visual saliency property of human visual system, the paper proposes a novel salient regions extraction method to model target appearance. In order to capture the salient and spatial information within this model, the method extracts a set of local salient descriptors based on covariance features from the target. Afterwards, an optimization problem is constructed with respect to the features of these salient regions, and the optimal target state is obtained by solving this problem using a gradient descent algorithm. Experiments on several challenging video sequences demonstrate the good performance of the proposed method compared with four state-of-art tracking methods.

This paper presents a novel formulation for object tracking. We model the second-order statistics of image regions and perform covariance matching under the variational level set framework. Specifically, covariance matrix is adopted as a visual object representation for partial differential equation (PDE) based contour tracking. Log-Euclidean calculus is used as a covariance distance metric instead of Euclidean distance which is unsuitable for measuring the similarities between covariance matrices, because the matrices typically lie on a non-Euclidean manifold. A novel image energy functional is formulated by minimizing the distance metrics between the candidate object region and a given template, and maximizing the ones between the background region and the template. The corresponding gradient flow is then derived according to a variational approach, enabling PDE-based visual tracking. Experiments on synthetic and real video sequences prove the validity of the proposed method.

In this paper, a novel 3D head model retrieval approach is proposed, in which only a single 2D face view query is required. The proposed approach will be important for multimedia application areas such as virtual world construction and game design, in which 3D virtual characters with a given set of facial features can be rapidly constructed based on 2D view queries, instead of having to generate each model anew. To achieve this objective, we construct an adaptive mapping through which each 2D view feature vector is associated with its corresponding 3D model feature vector. Given this estimated 3D model feature vector, similarity matching can then be performed in the 3D model feature space. To avoid the explicit specification of the complex relationship between the 2D and 3D feature spaces, a neural network approach is adopted in which the required mapping is implicitly specified through a set of training examples. In addition, for efficient feature representation, principal component analysis (PCA) is adopted to achieve dimensionality reduction for facilitating both the mapping construction and the similarity matching process. Since the linear nature of the original PCA formulation may not be adequate to capture the complex characteristics of 3D models, we also consider the adoption of its nonlinear counterpart, i.e., the so-called kernel PCA approach, in this work. Experimental results show that the proposed approach is capable of successfully retrieving the set of 3D models which are similar in appearance to a given 2D face view.

In this paper we present a novel method of using genetic algorithm (GA) to learn a graphical model which is used for human motion characterization. The modeling of human movements will involve a high dimensional joint probability density function. With this graphical model, the joint probability distribution can be decomposed into a number of low dimensional distributions which are represented as tree models and triangulated models. To automatically search for such a model from a database of cases is a NP-hard problem. We use GA to solve this problem, which can optimize both the ordering structure and the conditional independence relationship of the graphical model. The searched graphical models are used to classify different types of human motions. The experimental results demonstrate that, compared with a previous greedy search algorithm, the GA is more effective for optimization of the graphical model

In this paper, a novel 3D head model retrieval framework is proposed. First, kernel PCA is adopted both to reduce the data dimension and to extract features for model characterization. Second, based on the derived features, a hierarchical indexing structure for 3D model database is constructed using the hierarchical self organizing map (HSOM). Third, an efficient search approach is presented based on the established indexing structure that requires only feature matching between the query model and a small number of SOM nodes. The main advantages of our approach include high retrieval precision due to the discrimination capacity of kernel PCA, and low computation cost due to the hierarchical indexing structure and data dimension reduction. In addition, the topology-preserving property of HSOM also facilitates the exploration of the model database with the possibility of further knowledge discovery.

In this paper, we propose a novel 3D head model retrieval approach in which the queries are 2D face views instead of less readily available 3D head models. The basic idea is to characterize the corresponding relations between 2D view feature and 3D model feature based on a machine learning approach. Thus the subsequent feature matching can be carried out in 3D feature space. As an effective solution to regression problems, relevance vector machine is used in this paper to establish an association between 2D and 3D features. Experimental results show that our proposed 2D query based method is comparable with the direct 3D query based one.