Multiple kernel learning (MKL) is a principled approach for selecting and combining kernels for a given recognition task. A number of studies have shown that MKL is a useful tool for object recognition, where each image is represented by multiple sets of features and MKL is applied to combine different feature sets. We review the state-of-the-art for MKL, including different formulations and algorithms for solving the related optimization problems, with the focus on their applications to object recognition. One dilemma faced by practitioners interested in using MKL for object recognition is that different studies often provide conflicting results about the effectiveness and efficiency of MKL. To resolve this, we conduct extensive experiments on standard datasets to evaluate various approaches to MKL for object recognition. We argue that the seemingly contradictory conclusions offered by studies are due to different experimental setups. The conclusions of our study are: (i) given a sufficient number of training examples and feature/kernel types, MKL is more effective for object recognition than simple kernel combination (e.g., choosing the best performing kernel or average of kernels); and (ii) among the various approaches proposed for MKL, the sequential minimal optimization, semi-infinite programming, and level method based ones are computationally most efficient.

The objective in developing compact descriptors for visual image search is building an image retrieval system that works efficiently and effectively under bandwidth and memory constraints. Selecting local descriptors to be processed, and sending them to the server for matching is an integral part of such a system. One such image search and retrieval system is the Compact Descriptors for Visual Search (CDVS) standardization test model being developed by MPEG which has an efficient local descriptor selection criteria. However, all the existing selection parameters in CDVS are based on low-level features. In this paper, we propose two “mid-level” local descriptor selection criteria: Visual Meaning Score (VMS), and Visual Vocabulary Score (VVS) which can be seamlessly integrated into the existing CDVS framework. A mid-level criteria explicitly allows selection of local descriptors closer to a given set of images. Both VMS and VVS are based on visual words (patches) of images, and provide significant gains over the current CDVS standard in terms of matching accuracy, and have very low implementation cost.

Image matching and search is gaining significant commercial importance nowadays due to various applications it enables such as augmented reality, image-queries for internet search, etc. Many researchers have effectively used color information in an image to improve its matching accuracy. These techniques, however, cannot be directly used for large scale mobile visual search applications that pose strict constraints on the size of the extracted features, computational resources and the system accuracy. To overcome this limitation, we propose a new and effective technique to incorporate color information that can use the SIFT extraction technique. We conduct our experiments on a large dataset containing around 33, 000 images that is currently being investigated in the MPEG-Compact Descriptors for Visual Search Standard and show substantial improvement compared to baseline.

We consider a special type of multi-label learning where class assignments of training examples are incomplete. As an example, an instance whose true class assignment is (c1, c2, c3) is only assigned to class c1when it is used as a training sample. We refer to this problem as multi-label learning with incomplete class assignment. Incompletely labeled data is frequently encountered when the number of classes is very large (hundreds as in MIR Flickr dataset) or when there is a large ambiguity between classes (e.g., jet vs plane). In both cases, it is difficult for users to provide complete class assignments for objects. We propose a ranking based multi-label learning framework that explicitly addresses the challenge of learning from incompletely labeled data by exploiting the group lasso technique to combine the ranking errors. We present a learning algorithm that is empirically shown to be efficient for solving the related optimization problem. Our empirical study shows that the proposed framework is more effective than the state-of-the-art algorithms for multi-label learning in dealing with incompletely labeled data.

Consider a typical recommendation problem. A company has historical records of products sold to a large customer base. These records may be compactly represented as a sparse customer-times-product ``who-bought-what" binary matrix. Given this matrix, the goal is to build a model that provides recommendations for which products should be sold next to the existing customer base. Such problems may naturally be formulated as collaborative filtering tasks. However, this is a {\it one-class} setting, that is, the only known entries in the matrix are one-valued. If a customer has not bought a product yet, it does not imply that the customer has a low propensity to {\it potentially} be interested in that product. In the absence of entries explicitly labeled as negative examples, one may resort to considering unobserved customer-product pairs as either missing data or as surrogate negative instances. In this paper, we propose an approach to explicitly deal with this kind of ambiguity by instead treating the unobserved entries as optimization variables. These variables are optimized in conjunction with learning a weighted, low-rank non-negative matrix factorization (NMF) of the customer-product matrix, similar to how Transductive SVMs implement the low-density separation principle for semi-supervised learning. Experimental results show that our approach gives significantly better recommendations in comparison to various competing alternatives on one-class collaborative filtering tasks.

Recent studies have shown that multiple kernel learning is very effective for object recognition, leading to the popularity of kernel learning in computer vision problems. In this work, we develop an efficient algorithm for multi-label multiple kernel learning (ML-MKL). We assume that all the classes under consideration share the same combination of kernel functions, and the objective is to find the optimal kernel combination that benefits all the classes. Although several algorithms have been developed for ML-MKL, their computational cost is linear in the number of classes, making them unscalable when the number of classes is large, a challenge frequently encountered in visual object recognition. We address this computational challenge by developing a framework for ML-MKL that combines the worst-case analysis with stochastic approximation. Our analysis shows that the complexity of our algorithm is $O(m^{1/3}\sqrt{ln m})$, where $m$ is the number of classes. Empirical studies with object recognition show that while achieving similar classification accuracy, the proposed method is significantly more efficient than the state-of-the-art algorithms for ML-MKL.

Multi-label learning is useful in visual object recognition when several objects are present in an image. Conventional approaches implement multi-label learning as a set of binary classification problems, but they suffer from imbalanced data distributions when the number of classes is large. In this paper, we address multi-label learning with many classes via a ranking approach, termed multi-label ranking. Given a test image, the proposed scheme aims to order all the object classes such that the relevant classes are ranked higher than the irrelevant ones. We present an efficient algorithm for multi-label ranking based on the idea of block coordinate descent. The proposed algorithm is applied to visual object recognition. Empirical results on the PASCAL VOC 2006 and 2007 data sets show promising results in comparison to the state-of-the-art algorithms for multi-label learning.

Nonnegative matrix factorization (NMF) has been shown to be an efficient clustering tool. However, NMF¿s batch nature necessitates recomputation of whole basis set for new samples. Although NMF is a powerful content representation tool, this limits the use of NMF in online processing of large data sets. Another problem with NMF, like other partitional methods, is determining the actual number of clusters. Deciding the rank of the factorization is also critical since it has a significant effect on clustering performance. This paper introduces an NMF based incremental clustering algorithm which allows increasing number of clusters adaptively thus eliminates optimal rank selection problem. Test results obtained on large video data sets demonstrate that the proposed clustering scheme is capable of labeling linearly separable data as well as non-separable samples with a small false positive ratio.

Nonnegative matrix factorization (NMF) is a powerful decomposition tool which has been used in several content representation applications recently. However, there are some difficulties in implementing NMF in on-line video applications. This paper introduces an incremental NMF (INMF) without deviating from conventional NMF"s main objective function, which is minimizing the reconstruction error. The proposed algorithm is capable of modeling dynamic content of the video; thus controls contribution of the subsequent observations to the NMF representation properly. It is shown that the INMF preserves additive, parts-based representation capability of the NMF with a low computational load while offering dimension reduction. Experimental results are given to compare the reconstruction performances of the conventional and incremental NMF. In addition, video scene change detection and dynamic video content representation by INMF are investigated. Test results demonstrate that the INMF can be used as a powerful on-line factorization tool.