This paper addresses the problem of affine distortions caused by viewpoint changes for the application of image retrieval. We study how to expand the visual words from a query image for better retrieval recall without the sacrifice of retrieval precision and efficiency. Our main contribution is the building of visual dictionaries that retain the mapping relationships between visual words extracted from different viewpoints of the same object. Additionally, in each mapping rule we record the affine transformation in which the two visual words are related, as a compact code of viewpoints relationships. By analogizing the concepts of verb stem and verb tense in text, we use Visual Stems to denote visual words extracted from robust local patches, and record the relationships between their affine variants as visual stem mapping rules, including the geometric relationships coded as Geometric Tenses. In this way, our method augments original visual vocabulary with sufficient and accurate expansion information. In query phase, only the objects corresponding to the same visual stems and coherent geometric tense codes will be regarded as similar ones. Moreover, the mapping rules can be learned offline with only one sample for each object. Experiments show that our method can support efficient object retrieval with high recall, requiring little extra time and space cost over traditional visual vocabularies.

Progressive transmission is very effective to reduce retrieval latency in mobile visual search. However, the acceleration effects of existing progressive transmission strategies are often limited because of the neglect of geometric information in the query image. This paper proposes an effective and efficient geometric context-preserving progressive transmission method, which is suitable for mobile visual search. Here a query image is divided into blocks and local features in the same block are used as query units rather than a single feature. Since clustered features with geometric information are more discriminative, only a few of them could support correct matching with high precision. Thus our method significantly decreases the number of features needed for transmission, and dramatically reduces the retrieval latency. Experiments on Stanford dataset for mobile visual search show that, with comparable precision, we uses 43% less retrieval time than existing progressive transmission method. Moreover, we establish and release a large-scale image dataset called MVSBench which is more difficult and suitable for mobile visual search. It contains 75500 images and considers many variations like view change, blur, scale, illumination and rotation. MVSBench is another major contribution of this paper, and our method also outperforms other strategies on this dataset.