Since the labelling for the positive images/videos is ambiguous in weakly supervised segment annotation, negative mining‐based methods that only use the intra‐class information emerge. In these methods, negative instances are utilised to penalise unknown instances for ranking their likelihood of being an object, which can be considered as voting in terms of similarity. However, these methods (i) ignore the information contained in positive bags; (ii) only rank the likelihood but cannot generate an explicit decision function. In this study, the authors propose a voting scheme involving not only the definite negative instances but also the ambiguous positive instances to make use of the extra useful information in the weakly labelled positive bags. In the scheme, each instance votes for its label with a magnitude arising from the similarity, and the ambiguous positive instances are assigned soft labels that are iteratively updated during the voting. It overcomes the limitations of voting using only the negative bags. They also propose an expectation kernel density estimation algorithm to gain further insight into the voting mechanism. Experimental results demonstrate the superiority of the authors’ scheme beyond the baselines.

In this paper, we propose a multiple-instance discriminant analysis algorithm for weakly supervised segment annotation. We introduce a selection parameter for each image/video with weak labels and expect that it can sift out object regions from the background clutter to train a better transformation vector. The selection parameter and the transformation parameter are incorporated into a single objective function and optimized in an alternate way. The optimization is an iteration between the eigenvalue decomposition and a set of quadratic programming. We also integrate a regularization term into the objective function to formulate the spatial constraint of segments, which is ignored in ordinary multiple-instance learning methods. The algorithm is able to overcome the limitations that arise when applying ordinary multiple-instance methods to the task. The experimental results validate the effectiveness of our method.

Automatic object annotation for weakly labelled images/videos has attracted great research interests. In the literature, the idea of negative mining has been proposed for the task. Following existing works, the authors start with image/video over‐segmentation. With the assumption that the noisy segments in the concept images and the strongly labelled non‐concept segments are drawn from the same distribution, the authors plan to estimate the non‐concept distribution and apply it to the ambiguous segments to generate a concept ranking. Although this idea was proposed in existing work and was shown ineffective when combined with a naive kernel density estimation strategy, in this study, the authors explore improved density estimation techniques for the ranking and propose a kernel regression model whose parameters are estimated by a maximum likelihood estimation. Experimental results validate the effectiveness of their method.

In this paper, we aim at irregular-shape object localization under weak supervision. With over-segmentation, this task can be transformed into multiple-instance context. However, most multiple-instance learning methods only emphasize single most positive instance in a positive bag to optimize bag-level classification, and leads to imprecise or incomplete localization. To address this issue, we propose a scheme for instance annotation, where all of the positive instances are detected by labeling each instance in each positive bag. Inspired by the successful application of bag-of-words (BoW) to feature representation, we leverage it at instance-level to model the distributions of the positive class and negative class, and then incorporate the BoW learning and instance labeling in a single optimization formulation. We also demonstrate that the scheme is well suited to weakly supervised object localization of irregular-shape. Experimental results validate the effectiveness both for the problem of generic instance annotation and for the application of weakly supervised object localization compared to some existing methods.

Visual learning problems, such as object classification and action recognition, are typically approached using extensions of the popular bag-of-words (BoWs) model. Despite its great success, it is unclear what visual features the BoW model is learning. Which regions in the image or video are used to discriminate among classes? Which are the most discriminative visual words? Answering these questions is fundamental for understanding existing BoW models and inspiring better models for visual recognition. To answer these questions, this paper presents a method for feature selection and region selection in the visual BoW model. This allows for an intermediate visualization of the features and regions that are important for visual learning. The main idea is to assign latent weights to the features or regions, and jointly optimize these latent variables with the parameters of a classifier (e.g., support vector machine). There are four main benefits of our approach: 1) our approach accommodates non-linear additive kernels, such as the popular χ(2) and intersection kernel; 2) our approach is able to handle both regions in images and spatio-temporal regions in videos in a unified way; 3) the feature selection problem is convex, and both problems can be solved using a scalable reduced gradient method; and 4) we point out strong connections with multiple kernel learning and multiple instance learning approaches. Experimental results in the PASCAL VOC 2007, MSR Action Dataset II and YouTube illustrate the benefits of our approach.

In this paper, we investigate the problem of weakly supervised object localization in images. For such a problem, the goal is to predict the locations of objects in test images while the labels of the training images are given at image-level. That means a label only indicates whether an image contains objects or not, but does not provide the exact locations of the objects. We propose to address this problem using Maximal Entropy Random Walk (MERW). Specifically, we first train a linear SVM classifier with the weakly labeled data. Based on bag-of-words feature representation, the response of a region to the linear SVM classifier can be formulated as the sum of the feature-weights within the region. For a test image, by properly constructing a graph on the feature-points, the stationary distribution of a MERW can indicate the region with the densest positive feature-weights, and thus provides a probabilistic object localization. Experiments compared with state-of-the-art methods on two datasets validate the performance of our method.

How to build a compact and informative training data set autonomously is crucial for many real-world learning tasks, especially those with large amount of unlabeled data and high cost of labeling. Active learning aims to address this problem by asking queries in a smart way. Two main scenarios of querying considered in the literature are query synthesis and pool-based sampling. Since in many cases synthesized queries are meaningless or difficult for human to label, more efforts have been devoted to pool-based sampling in recent years. However, in pool-based active learning, querying requires evaluating every unlabeled data point in the pool, which is usually very time-consuming. By contrast, query synthesis has clear advantage on querying time, which is independent of the pool size. In this paper, we propose a novel framework combining query synthesis and pool-based sampling to accelerate the learning process and overcome the current limitation of query synthesis. The basic idea is to select the data point nearest to the synthesized query as the query point. We also provide two simple strategies for synthesizing informative queries. Moreover, to further speed up querying, we employ clustering techniques on the whole data set to construct a representative unlabeled data pool based on cluster centers. Experiments on several real-world data sets show that our methods have distinct advantages in time complexity and similar performance compared to pool-based uncertainty sampling methods.