Figure 1: SiCo Overview.Our system begins by prompting users to upload an image of themselves and provide their actual sizes for tops and bottoms.Users then proceed to the garment selection step, where they can choose different sizes for an item (Steps 1 and 3) and visualize how it would look on them (Steps 2 and 4, respectively).In addition to trying on items individually, users can also style multiple items together (Steps 5 and 6).The features in our system enable users to interact with garment sizes and fits more intuitively, providing valuable insights to help them make informed decisions about which garments and sizes to purchase.Garment images are sourced from the DressCode dataset [41].Model image © Pexels.

Although natural language instructions offer an intuitive way to guide automated image editing, deep- learning models often struggle to achieve high- quality results, largely due to the difficulty of creating large, high- quality training datasets. To do this, previous approaches have typically relied on text- to- image (T2I) generative models to produce pairs of original and edited images that simulate the input/output of an instruction- guided image- editing model. However, these image pairs often fail to align with the specified edit instructions due to the limitations of T2I models, which negatively impacts models trained on such datasets. To address this, we present Instruct- CLIP (I- CLIP), a self-supervised method that learns the semantic changes between original and edited images to refine and better align the instructions in existing datasets. Furthermore, we adapt Instruct- CLIP to handle noisy latent images and diffusion time-steps so that it can be used to train latent diffusion models (LDMs) and efficiently enforce alignment between the edit instruction and the image changes in latent space at any step of the diffusion pipeline. We use Instruct- CLIP to correct the InstructPix2Pix dataset and get over 120K refined samples we then use to fine- tune their model, guided by our novel I- CLIP- based loss function. The resulting model can produce edits that are more aligned with the given instructions. Our code and dataset are available at https://github.com/SherryXTChen/Instruct-CLIP.git.

Despite many attempts to leverage pre-trained text-to-image models (T2I) like Stable Diffusion (SD) [25] for controllable image editing, producing good predictable results remains a challenge. Previous approaches have focused on either fine-tuning pre-trained T2I models on specific datasets to generate certain kinds of images (e.g., with a specific object or person), or on optimizing the weights, text prompts, and/or learning features for each input image in an attempt to coax the image generator to produce the desired result. However, these approaches all have shortcomings and fail to produce good results in a predictable and controllable manner. To address this problem, we present TiNO-Edit, an SD-based method that focuses on optimizing the noise patterns and diffusion timesteps during editing, something previously unexplored in the liter-ature. With this simple change, we are able to generate results that both better align with the original images and reflect the desired result. Furthermore, we propose a set of new loss functions that operate in the latent domain of SD, greatly speeding up the optimization when compared to prior losses, which operate in the pixel domain. Our method can be easily applied to variations of SD including Textual Inversion [13] and DreamBooth [27] that encode new concepts and incorporate them into the edited results. We present a host of image-editing capabilities enabled by our approach. Our code is publicly available at https://github.com//SherryXTChen/TiNO-Edit.

The DETR object detection approach applies the Transformer encoder and decoder architecture to detect objects from images and achieves promising performance. In this paper, we present a simple approach to address the main problem of DETR, the slow convergence, by using representation learning technique. In this approach, we detect an object bounding box as a pair of keypoints, the top-left corner and the center, using two decoders. By detecting objects as paired keypoints, the model builds up a joint classification and pair association on the output queries from two decoders. For the pair association we propose utilizing contrastive self-supervised learning algorithm without requiring specialized architecture. Experimental results on MS COCO dataset show that Pair DETR can reduce training epoch by 10x from original DETR and 1.5x from Conditional DETR, while having consistently higher Average Precision scores.