Visual codewords assignment plays an important role in many Bag of Features (BoF) models for image understanding and visual recognition. It allocates image descriptors to the most similar codewords in the pre-configured visual dictionary to generate descriptive histogram for the consequent categorization. Nevertheless, existing assignment approaches, e.g. nearest neighbors strategy and Gaussian similarity, suffer from two problems:1) too strong Euclidean assumption and 2) neglecting the label information of the local features. Accordingly, in this paper, we propose an assignment method to simultaneously consider the above two issues in a unified model via graph learning and information theoretic criterions. For learning, the proposed model can be efficiently solved in a closed-form with the reasonable graph topology invariant approximation. Moreover, the learned projections enable us to extend the assignment ability to the out-of-sample visual features beyond the initial training graph. Experiments on our own manifold dataset and two benchmarks verify the effectiveness of the proposed graph assignment method.

We address the problem of measuring the distance between two subspaces, each of which is spanned by an image set. In the existing methods, only the orthonormal basis is used to represent the subspace. However, the images are usually distributed in a limited area, rather than the whole subspace. Therefore, the characteristics of the distribution should also be considered. In this letter, a weighted subspace distance (WSD) is proposed, in which the principal component values of the data set are adopted to calculate the weights. Experimental results on object recognition and retrieval with image sets demonstrate the effectiveness of our proposal.

We propose to improve the effectiveness and scalability of graph based manifold ranking methods in image retrieval applications by emphasizing reliable images while damping the effect of noisy or irregular ones. Label information is firstly passed between most reliable data points, then propagated to less reliable ones on manifold structure. By treating these images inequivalently, undesirable effect of noisy samples is greatly reduced, thus effectiveness of manifold ranking algorithms is enhanced. Also, graph size in terms of number of nodes and edges is dramatically reduced, resulting in a great speed-up of the algorithm. Our experiment on real world image data set demonstrates the effectiveness of the proposed approach.

Content-based image retrieval (CBIR) has been an active research topic in the last decade. As one of the promising approaches, graph-based semi-supervised learning has attracted many researchers. However, while the related work mainly focused on global visual features, little attention has been paid to region-based image retrieval (RBIR). In this paper, a framework based on multilabel neighborhood propagation is proposed for RBIR, which can be characterized by three key properties: (1) For graph construction, in order to determine the edge weights robustly and automatically, mixture distribution is introduced into the Earth mover's distance (EMD) and a linear programming framework is involved. (2) Multiple low-level labels for each image can be obtained based on a generative model, and the correlations among different labels are explored when the labels are propagated simultaneously on the weighted graph. (3) By introducing multilayer semantic representation (MSR) and support vector machine (SVM) into the long-term learning, more exact weighted graph for label propagation and more meaningful high-level labels to describe the images can be calculated. Experimental results, including comparisons with the state-of-the-art retrieval systems, demonstrate the effectiveness of our proposal.

Label propagation and manifold ranking have been successfully adopted in content-based image retrieval (CBIR) in recent years. However, while the global low-level features are widely utilized in current systems, region-based features have received little attention. In this paper, a novel transductive framework based on correlated probabilistic label propagation is proposed for region-based images retrieval (RBIR), which can be characterized by three key properties: (1) unified feature matching (UFM) is chosen to measure the similarity between two segmented images. (2) To represent the segmented images in a uniform feature space, a generative model is adopted and the probabilistic labels of each image can be obtained. (3) In the retrieval process, multiple probabilistic labels of training samples are propagated simultaneously on the weighted graph, and the correlation among different labels are explored. Experimental results on 10000 images show that our algorithm can greatly improve the retrieval performance of the RBIR system.

Content-based image retrieval (CBIR) has been more and more important in the last decade, and the gap between high-level semantic concepts and low-level visual features hinders further performance improvement. The problem of online feature selection is critical to really bridge this gap. In this paper, we investigate online feature selection in the relevance feedback learning process to improve the retrieval performance of the region-based image retrieval system. Our contributions are mainly in three areas. 1) A novel feature selection criterion is proposed, which is based on the psychological similarity between the positive and negative training sets. 2) An effective online feature selection algorithm is implemented in a boosting manner to select the most representative features for the current query concept and combine classifiers constructed over the selected features to retrieve images. 3) To apply the proposed feature selection method in region-based image retrieval systems, we propose a novel region-based representation to describe images in a uniform feature space with real-valued fuzzy features. Our system is suitable for online relevance feedback learning in CBIR by meeting the three requirements: learning with small size training set, the intrinsic asymmetry property of training samples, and the fast response requirement. Extensive experiments, including comparisons with many state-of-the-arts, show the effectiveness of our algorithm in improving the retrieval performance and saving the processing time.

Region-based image retrieval and relevance feedback are two important methods to bridge the gap between the low-level visual features and the high-level semantic concepts in content-based image retrieval. In this paper, we address the issue of introducing the relevance feedback mechanism into the region-based image retrieval with online feature selection during each feedback round. Our contribution is two-fold. (1) A novel region-based image representation is proposed. Based on a generative model, a fuzzy codebook is extracted from the original region-based features, which represents the images in a uniform real-value feature space. (2) A feature selection criterion is developed and an effective relevance feedback algorithm is implemented in a boosting manner to simultaneously select the optimal features from the fuzzy codebook, and generate a strong ensemble classifier over the selected features. Experimental results show that the proposed scheme can substantially improve the retrieval performance of the region-based image retrieval system.

Content-based image retrieval can be viewed as a classification problem, and the small sample size leaning difficulty makes it difficult for most CBIR classifiers to get satisfactory performance. In this paper, using the SVM classifier as the component classifier, the method of ensemble of classifiers is incorporated into the relevance feedback process to alleviate this problem from two aspects: (1) within each feedback round, multiple parallel component classifiers are constructed, one over one feature subspace individually, and then are merged together to get an ensemble classifier; (2) during feedback rounds, a boosting method is incorporated to sequentially combine the component classifiers over each feature subspace respectively, which further improves the classification result. Experiments over 5000 images show that the proposed method can improve the retrieval performance consistently, without loss of efficiency.

This paper proposes an improved rate allocation method, based on a sliding window, for FGS (fine granular scalability) video bit-streams. It can allocate the rate for every frame in the FGS video bit-stream rapidly, to smooth video quality under dynamical channel conditions. The paper formulates the FGS rate allocation problem under time-varying channel conditions using a sliding window to record the available network bandwidth. Under the constrained bandwidth, the proposed method first allocates rates in advance and creates a reference window by solving the problem of constant quality constrained FGS rate allocation. Then by refreshing the reference and sliding windows, the rate allocation of other frames can be solved rapidly. Theoretical analysis and simulation results for the method demonstrate its low complexity and high performance.

Long-term relevance feedback learning is an important learning mechanism in content-based image retrieval. In this paper, our work has two contributions: (1) A multilayer semantic representation (MSR) is proposed and an algorithm is implemented to automatically build the MSR for image database through long-term relevance feedback learning. (2) The accumulated MSR is incorporated with the short-term feedback learning to help subsequent users' retrieval. The MSR memorizes the multicorrelation among images and integrates these memories to build hidden semantic concepts for images, which are distributed in multiple semantic layers. In experiment, an MSR is built based on the real retrieval from 10 different users, which can precisely describe the hidden concepts underlying images and help to bridge the gap between high-level concepts and low-level features and thus improve the retrieval performance significantly.

Extraction quality and speed are two fundamental problems for semantic video object extraction. The paper introduces a novel fast-speed tracking algorithm for extracting semantic video objects from image sequences. First, the subregion that covers the contour of a semantic video object is extracted by using motion prediction and mathematical morphology operators to generate the inner and outer contour of the subregion. Then, an optimized tracking scheme is employed to track this particular subregion instead of the whole image. Results on real sequences show that this method greatly improves the processing speed compared to earlier tracking algorithms and that it keeps the quality of the extracted object. Therefore, it is a promising approach for real time processing systems based on video objects.