Image retrieval aims to find similar images given the query. Most of existing retrieval works are based on the pre-trained model of single-label image classification. In practice, the query usually contains more than one instance, and the single label is far from enough for fully depicting the attributes of an open-world image. Due to the complicated similarity relationships between multiple semantics, the multi-label image retrieval task is not so well solved as the single-label task. In this work, we propose a two-stage pure Transformer model for multi-label image retrieval, which leverages a Transformer encoder to exploit the complex dependencies among visual features and labels. Except for the Transformer encoder, the image feature embedding module is also based on Transformer, so that the optimal model weights could be learned in an end-to-end manner. To be specific, inputs of the Transformer encoder mainly consist of an Vision Transformer branch and a label embedding branch, which generates suitable image features and label descriptions, respectively. Given an input set of visual features and text labels, the developed Transformer encoder could be accordingly optimized in the training stage with compressed multi-label output layer. In order to obtain sufficient outputs to accurately find images containing similar semantics with the query from the database, we adjust the network by removing the last fully connected layer in the retrieval stage. Specially, images and labels are used for the training stage in a randomly masked manner to enhance the model performance, and no labels are visible in the content-based image retrieval stage. Comprehensive experiments are performed on three multi-label datasets including MS-COCO, NUS-WIDE and VOC2007, demonstrating promising results of our proposed method against the state-of-the-arts for multi-label image retrieval.

Pattern images are artificially designed images that possess distinctiveness in their elements, styles, and arrangements. With the ever-growing number of pattern images, pattern image retrieval emerges as a promising technique with significant potential for commercial and industrial applications, such as fashion and home decoration, facilitating rapid identification of preferred print patterns by users. The main purpose of multi-label pattern image retrieval is to effectively represent and match images with their corresponding labels. Compared to conventional image retrieval, multi-label pattern image retrieval faces greater challenges due to the richer semantic information contained within the abstract print patterns and the complex relationships between multiple labels. To tackle these challenges, we propose a model specifically designed for multi-label pattern image retrieval, called Tran-GCN. Our proposed model is built upon a Transformer-based autoregressive architecture, which leverages image information to guide the exploration of correlations between different labels through the textual modality. By utilizing this correlation information, we construct a graph convolutional network (GCN) model to further enhance the correlations between image and label representations. To be more specific, our Tran-GCN model utilizes a cross-modal attention mechanisms at each layer to effectively aggregate visual features from the input image and update label semantics through residual connections. The GCN module is updated based on the correlation between textual features, as represented in a relationship matrix. Extensive experiments on two widely used public visual benchmarks, MS-COCO and NUS-WIDE, as well as a multi-label pattern image dataset, Pattern 2, consistently demonstrate the ability of our proposed Tran-GCN model for general use and its superior performance in multi-label pattern image retrieval tasks as well.