Automatic real image editing offers unprecedented freedom to modify the appearance of the image or to edit a few objects through natural language. Recent scalable model families such as diffusion models have showcased remarkable proficiency in editing highly realistic images due to the introduction of vast amounts of training data and large pretrained language models. However, these large diffusion models require iterative evaluation that would significantly hinder the pace of image editing. Moreover, the pioneering work in this field necessitates the learning of a unique textual token that corresponds to each input image, or a group of images containing the same object, leading to the generation of redundant and fragmented models. Given the aforementioned problems, we suggest a novel Large pretrained models Assistant text-guided image Editing adversarial Network (LAE-Net) in this paper. More concretely, we introduce a deep semantic editing network to globally transfer text information among different isolated editing blocks, which would extract features from the source image to differentiate text-required areas from text-irrelevant ones. Furthermore, based on idea that the multi-modal CLIP model, leveraging vision-language alignment, captures comprehensive global semantic cues, whereas the vision-centric DINO model specializes in delivering intricate, fine-grained pixel-level details, the powerful discriminator of LAE-Net is designed by harnessing the visual embeddings derived from both the CLIP and DINO models separately to boost the visual discriminant capability and facilitate training a strong generator for conditioning image generation. Comprehensive experimental evaluations show that our LAE-Net not only delivers outstanding performance but also surpasses several cutting-edge models.

Reconstructing a 3D hand from a single RGB image is a very challenging task. Most of the existing Transformer-based 3D hand reconstructing methods do not fully consider the local spatial information from low-level image features, which would be crucial for capturing fine details and accurate shapes of the hand. Consequently, this oversight often leads to reconstructed hands that lack the precision and realism necessary for many applications, such as augmented reality, and hand gesture recognition. To address this limitation, in this paper, we propose a novel and efficient method named HybridMETRO to both utilize low-level and high-level image features for accurate reconstructing 3D hand pose and mesh vertices from a single RGB image. Specifically, we introduce the deformable attention into the encoder of Transformer, making it no longer limited by the length of the image feature sequence. Based on the above mechanism, we further propose an interleaved updating multi-scale feature encoder to fuse low-level and high-level features. Moreover, we incorporate the Graph Convolutional Residual (GCR) module to build a novel decoder to capture explicit semantic connections between grid vertices and thus improve spatial locality of extracted features. Experimental results demonstrate that, when compared with state-of-the-art methods, our proposed HybridMETRO could achieve better performance with significantly smaller model parameters that are about half of METRO’s and a quarter of HandOccNet’s.

Abstract Content‐based image retrieval (CBIR) is the problem of searching for items in an image database that are similar to the query image. Most of the existing image retrieval methods are trained based on metric learning loss functions (e.g. contrastive loss or triplet loss), however, which require the use of hard sample mining strategies (HMS) to better train the model. The HMS implies that picking out hard positive or negative samples increases the complexity of model training and requires a large amount of additional training time. To address this issue, lessons from recent work are leveraged on representation learning and a model called GS is proposed that combines the state‐of‐the‐art Generalized‐Mean (GeM) pooling and the smoothed average precision (AP). The entire network can be learned end‐to‐end by approximating the non‐differentiable AP function to a differentiable one‐without mining hard samples, only image‐level annotations. A model named GSA is also presented which achieves excellent retrieval performance jointly trained by two various loss functions. Experimental results validate the effectiveness of the proposed approach and demonstrate the competitive performance on a common standard image retrieval dataset (Revisited Oxford and Paris).

The state of the art of technology for near-duplicate image retrieval is mostly based on the Bag-of-Visual-Words model. However, visual words are easy to result in mismatches because of quantization errors of the local features the words represent. In order to improve the precision of visual words matching, contextual descriptors are designed to strengthen their discriminative power and measure the contextual similarity of visual words. This paper presents a new contextual descriptor that measures the contextual similarity of visual words to immediately discard the mismatches and reduce the count of candidate images. The new contextual descriptor encodes the relationships of dominant orientation and spatial position between the referential visual words and their context. Experimental results on benchmark Copydays dataset demonstrate its efficiency and effectiveness for near-duplicate image retrieval.

The Bag-of-Visual-Words model has become a popular model in image retrieval and computer vision. But when the local features of the Interest Points (IPs) are transformed into visual words in this model, the discriminative power of the local features are reduced or compromised. To address this issue, in this paper, we propose a novel contextual descriptor for local features to improve its discriminative power. The proposed contextual descriptors encode the dominant orientation and directional relationships between the reference interest point (IP) and its context. A compact Boolean array is used to represent these contextual descriptors. Our experimental results show that the proposed contextual descriptors are more robust and compact than the existing contextual descriptors, and improve the matching accuracy of visual words, thus make the Bag-of-Visual-Words model become more suitable for image retrieval and computer vision tasks.

This paper presents a visual attention model for dynamic scenes based on motion features. The aim is to obtain the region of interest in accordance with the observer's attention in a multiple moving objects situation. Motion speed and motion orientation are selected as the motion features, which make the processing in the model similar to natural observation. A block matching algorithm and a “center-surround” operation are used to extract features. A motion saliency map is created by the integration of a motion speed conspicuity map and a motion orientation conspicuity map. The experiment shows that the model performs an object-based selection which is effective in complex scenes with efficient background noise suppression.