In modern fashion design field, it is a mainstream practice to generate clothing images by combining sketch and text. However, the image quality generated by existing multimodal methods combining sketches and text descriptions is suboptimal, as the clothing in the generated image often lacks contour accuracy and stylistic coherence. In this paper, we present STCGen, an advanced multimodal framework that uses both sketches and text to generate clothing images with improved contours and more consistent style. First, we introduce the sketch prior embedding module, which processes sketches to extract key structural features and ensure the consistency of contours, thereby enhancing image details. Second, we propose a cross space attention mechanism to address the issue of text information loss and ensure stylistic consistency, thereby enhancing overall image coherence. Finally, we propose a network simplification scheme to reduce complexity without compromising the quality of resulting images. Experimental results demonstrate that our method excels in generating high-fidelity clothing images.

Trackers based on the region proposal network (RPN) have garnered extensive attention within object tracking. However, current RPN trackers utilizing the ResNet as the feature extraction network only employ its local convolution operation to extract the image features, thereby constraining the model’s comprehension of the global contextual information. To address this issue, this paper employs a Vision Transformer (ViT) to construct global relations across the entire sequence features, enhancing the model’s capability to represent global features. Additionally, traditional single-stage RPN trackers generate candidate boxes through a coarse regression process. Thus, the candidate boxes may only partially cover the object or include excessive background information, diminishing the model’s accuracy in object localization. Consequently, we use a multi-stage RPN to adjust the anchor boxes in a cascade RPN manner. Moreover, traditional multi-stage RPN exists the misalignment problem between the anchor boxes and image features. To further optimize the multi-stage RPN, this paper employs adaptive convolution to align features with anchor boxes. Our tracking method achieves state-of-the-art results on five common tracking benchmark datasets. Specifically, the CasRPN achieves an AUC score of 85.3% on the large-scale Trackingnet dataset.

High-fidelity garment reconstruction is essential for various applications such as garment design and virtual try-on. While image-based reconstruction methods have made significant progress with deep generative models, generating 3D models from hand-drawn sketches to meet design intentions remains challenging. One of the main obstacles is the limited availability of large-scale 3D garment models accompanied by corresponding sketches. To address this issue, we propose SGM, a comprehensive dataset comprising 656 garment models categorized into short and long sleeves. Each garment model in SGM is accompanied by four types of rendered images and a series of UDF values. Furthermore, we introduce a novel baseline approach for sketch-based garment reconstruction using an end-to-end generative network capable of generating garment models from single hand-drawn sketches. Extensive experimental results highlight the significance and value of our proposed dataset and method. We plan to make SGM publicly available upon publication.

The stitching of borehole images has an important predictive role in safety analysis in the field of geotechnical engineering and intelligent geological exploration. Applying traditional image stitching methods that designed specifically for high-resolution images to low-resolution images will lead to blurred stitching results, stitching seams, fewer matched feature points and difficulties in massive image stitching. To address these problems, we propose an autoencoder-based coarse-to-fine feature extraction network, which can extract image features with high semantic and improves the accuracy of the feature point matching. Besides, we design a cross-cycle Transformer-based image stitching framework, which increase the number of matching feature points by Cross-QuadTree attention and stitch image by affine transformation. Experimental results show that the proposed method can effectively stitch low-resolution geotechnical borehole images with satisfactory visual quality.

Fashion style learning is the basis for many tasks in fashion AI, such as clothing recommendations, fashion trend analysis and popularity prediction. Most of the existing methods rely on the quality and quantity of the annotations. This paper proposes an efficient two-step unsupervised fashion style learning framework with "Fashion Jigsaw" task and centroid-based density clustering algorithm. First, we design the "Fashion Jigsaw" unsupervised learning task according to the distribution of fashion elements in full-body fashion images. By splitting and recovering fashion images, we pre-train a model that can extract both intra-image and inter-image information. Second, we propose a centroid-based density clustering algorithm and introduce the concept of "centroid" to cluster fashion image features and represent fashion styles. Meanwhile, we keep the noise features to discover the newly sprouted fashion styles. Experiment results demonstrate the effectiveness of our proposed method.

As one of the specific vision tasks, person re-identification has become a prevalent research topic in the field of multimedia and computer vision. However, existing feature extraction methods, originating from the quality of the bounding boxes which could cause the inhomogeneity and incoherence of person representation for cluttered backgrounds, are difficult to adapt the challenges of the harsh real-world scenarios. This study develops a Triplet person Appearances Parsing Framework (TAPF) which eliminates the surrounding interference factors of bounding boxes for person re-identification. The framework consists of a triplet person parsing network and an integration mechanism for person local and global appearance information. Concretely, the triplet parsing network includes a channel parsing module, a position parsing module and a color parsing module, which are used to extract the person channel parsing descriptor, regional descriptor and color perception descriptor, respectively. Then, a local and global flatten gaussian operations are performed to integrate the person appearance parsing descriptors to obtain more discriminative features for the person representation. The experimental results have been conducted to validate our proposed algorithm can achieve a better performance for person re-identification on several public datasets, i.e., VIPeR and Market-1501, respectively.

Due to the nonlinear deformation of nonrigid and nonuniform tissues, it is challenging to accurately measure the displacements of feature points distributed on the inner parts, boundaries, and separatrices of tissue layers. To address this challenge, we propose a feature point matching technique called RAPID to measure MR 2D slice deformation of nonuniform and nonrigid biological tissues. We propose to use the covariance of several neighboring point statistics computed around a keypoint, as the keypoint descriptor. Inspired by the kernel methods, we advocate adopting a Riemannian pseudo kernel to map SPD matrices to a high dimensional Hilbert space, where the Euclidean geometry applies. We compare our RAPID with two existing schemes (i.e., SIFT and SURF). Our experimental results show that our RAPID is superior to SIFT and SURF, because the benefits offered by RAPID are two-fold. First, our RAPID increases the number of matched data points. Second, RAPID substantially improves the key-point matching accuracy of SIFT and SURF.