Bag-of-visual words (BoW) representation has been widely used in the large scale image retrieval. Though efficient, it ignores the geometric correlation among visual words, whereas the geometric verification has demonstrated its effectiveness in image retrieval. In this paper, we propose a new representation - visual group, to improve the retrieval precision by grouping the geometrically related features based on the inclusion relationship between features at different scales. A visual group consists of a master feature and several member features covered by the master feature. The geometric constraint inside each group is introduced into visual group matching for efficient geometric verification. Experimental evaluation on the dataset Oxford5K+Flickr1M shows that our visual group based image search approach outperforms BoW and the state-of-the-art visual phrase based schemes.

Although it has been extensively studied for many years, automatic image annotation is still a challenging problem. Recently, data-driven approaches have demonstrated their great success to image auto-annotation. Such approaches leverage abundant partially annotated web images to annotate an uncaptioned image. Specifically, they first retrieve a group of visually closely similar images given an uncaptioned image as a query, then figure out meaningful phrases from the surrounding texts of the image search results. Since the surrounding texts are generally noisy, how to effectively mine meaningful phrases is crucial for the success of such approaches. We propose a mixture modeling approach which assumes that a tag is generated from a convex combination of topics. Different from a typical topic modeling approach like LDA, topics in our approach are explicitly learnt from a definitive catalog of the Web, i.e. the Open Directory Project (ODP). Compared with previous works, it has two advantages: Firstly, it uses an open vocabulary rather than a limited one defined by a training set. Secondly, it is efficient for real-time annotation. Experimental results conducted on two billion web images show the efficiency and effectiveness of the proposed approach.

Instantly sharing captured landmark images is becoming fashionable, much like when you write a blog or chat with friends by mobile phone. However, real-time transmission of high-resolution images poses a significant challenge to contemporary mobile networks. Either long delays in transmission or largely reduced image resolution can lead to bad user experience. In this paper, we propose a novel mobile-cloud scheme IMShare to enable instant sharing of high-resolution images. On the mobile side, high-resolution images are described by their thumbnails and SIFT (Scale-Invariant Feature Transform) descriptors. After compression, data sent by mobile phones can be reduced to an average of 2.6 kilobytes (KB) per mega pixel. On the cloud side, high-resolution images are reproduced from a large-scale image database by retrieving partial duplicate images by SIFT descriptors and stitching corresponding image patches together under the guidance of the thumbnails. IMShare is the first scheme to demonstrate that not only visually pleasant images can be reconstructed using this mobile-cloud method but also the reconstruction can be done in seconds using parallel computing. Our user study of a half million images in a database shows that the proposed IMShare significantly outperforms the current method on subjective quality.