In this paper, we propose a simple but effective semantic-based aggregation (SBA) method. The proposed SBA utilizes the discriminative filters of deep convolutional layers as semantic detectors. Moreover, we propose the effective unsupervised strategy to select some semantic detectors to generate the "soft region proposals," which highlight certain discriminative pattern of objects and suppress the noise of background. The final global SBA representation could then be acquired by aggregating the regional representations weighted by the selected "soft region proposals" corresponding to various semantic content. Our unsupervised SBA is easy to generalize and achieves excellent performance on various tasks. We conduct comprehensive experiments and show that our unsupervised SBA outperforms the state-of-the-art unsupervised and supervised aggregation methods on image retrieval, place recognition, and cloud classification.

Existing manifold learning methods are not appropriate for image retrieval tasks, because most of them are unable to process query images and they have much greater computational cost especially for large-scale database. Therefore, we propose the iterative manifold embedding (IME) layer, of which the weights are learned offline by an unsupervised strategy, to explore the intrinsic manifolds by incomplete data. On the large-scale database that contains 27 000 images, the IME layer is more than 120 times faster than other manifold learning methods to embed the original representations at query time. We embed the original descriptors of database images that lie on manifold in a high-dimensional space into manifold-based representations iteratively to generate the IME representations in an offline learning stage. According to the original descriptors and the IME representations of database images, we estimate the weights of the IME layer by ridge regression. In the online retrieval stage, we employ the IME layer to map the original representation of a query image with an ignorable time cost (2 ms per image). We experiment on five public standard datasets for image retrieval. The proposed IME layer significantly outperforms the related dimension reduction methods and manifold learning methods. Without postprocessing, our IME layer achieves a boost in the performance of state-of-the-art image retrieval methods with postprocessing on most datasets, and needs less computational cost. The code is available at https://github.com/XJhaoren/IME_layer.

In this paper, we propose a simple but effective semantic part-based weighting aggregation (PWA) for image retrieval. The proposed PWA utilizes the discriminative filters of deep convolutional layers as part detectors. Moreover, we propose the effective unsupervised strategy to select some part detectors to generate the "probabilistic proposals," which highlight certain discriminative parts of objects and suppress the noise of background. The final global PWA representation could then be acquired by aggregating the regional representations weighted by the selected "probabilistic proposals" corresponding to various semantic content. We conduct comprehensive experiments on four standard datasets and show that our unsupervised PWA outperforms the state-of-the-art unsupervised and supervised aggregation methods.

Several recent works interpret convolutional features produced by deep convolutional neural networks as local descriptors. Existing high-dimensional aggregation based methods, e.g., Fisher Vector (FV) obtain inferior performance to pooling based methods in most situations, and we observe that it is mainly caused by the ignorance of spatial weights. In this paper, we propose a novel method named spatial weighted Fisher Vector (SWFV) to enhance the representation of FV by injecting the spatial weight map to FV. In addition, we further analyze the distribution of spatial weights and propose truncated spatial weighted FV (TSWFV). Experimental results on two benchmark datasets demonstrate that the two proposed methods achieve competitive results compared with other global representation based methods.

This letter presents a novel approach for logo retrieval. Considering the fact that logo only occupies a small portion of an image, we apply Faster R-CNN to detect logo proposals first, and then use a two-step pooling strategy with adaptive weight to obtain an accurate global signature. The adaptive weighted pooling method can effectively balance the recall and precision of proposals by incorporating the probability of each proposal being a logo. Experimental results show that the proposed method interprets the similarity between query and database image more accurately and achieves state of the art performance.

Characters, which are man-made symbols composed of strokes arranged in a certain structure, could provide semantic information and play an indispensable role in our daily life. In this paper, we try to make use of the intrinsic characteristics of characters and explore the stroke and structure-based methods for character recognition. First, we introduce two existing part-based models to recognize characters by detecting the elastic strokelike parts. In order to utilize strokes of various scales, we propose to learn the discriminative multi-scale stroke detector-based representation (DMSDR) for characters. However, the part-based models and DMSDR need to manually label the parts or key points for training. In order to learn the discriminative stroke detectors automatically, we further propose the discriminative spatiality embedded dictionary learning-based representation (DSEDR) for character recognition. We make a comparative study of the performance of the tree-structured model (TSM), mixtures-of-parts TSM, DMSDR, and DSEDR for character recognition on three challenging scene character recognition (SCR) data sets as well as two handwritten digits recognition data sets. A series of experiments is done on these data sets with various experimental setup. The experimental results demonstrate the suitability of stroke detector-based models for recognizing characters with deformations and distortions, especially in the case of limited training samples.