This paper presents a nonparametric scene parsing approach that improves the overall accuracy, as well as the coverage of foreground classes in scene images. We first improve the label likelihood estimates at superpixels by merging likelihood scores from different probabilistic classifiers. This boosts the classification performance and enriches the representation of less-represented classes. Our second contribution consists of incorporating semantic context in the parsing process through global label costs. Our method does not rely on image retrieval sets but rather assigns a global likelihood estimate to each label, which is plugged into the overall energy function. We evaluate our system on two large-scale datasets, SIFTflow and LMSun. We achieve state-of-the-art performance on the SIFTflow dataset and near-record results on LMSun.

Manually creating an object category dataset requires a lot of hard work and wastes a large amount of time. Having an automatic means for collecting images that represent different objects is crucial for the scalable and practical expansion of these datasets. In this work, a methodology to automatically re-rank the images returned from a web search engine is proposed to improve the precision of the retrieved results. The proposed system works in an incremental way to improve the learnt object model and achieve better precision in each iteration. Images along with their meta data are ranked, then re-filtered based on their textual and visual features to produce a robust set of seed images. These images are used in learning weighted distances between the images which are used to incrementally expand the collected dataset. Using our method, we automatically gather very large object category datasets. We also improve the image ranking performance of the retrieved results over web search engines and other batch methods.