Navve Wasserman, Roi Pony, Oshri Naparstek, Adi Raz Goldfarb, Eli Schwartz, Udi Barzelay, Leonid Karlinsky. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Image editing has advanced significantly with the introduction of text-conditioned diffusion models. Despite this progress, seamlessly adding objects to images based on textual instructions without requiring user-provided input masks remains a challenge. We address this by leveraging the insight that removing objects (Inpaint) is significantly simpler than its inverse process of adding them (Paint), attributed to inpainting models that benefit from segmentation mask guidance. Capitalizing on this realization, by implementing an automated and extensive pipeline, we curate a filtered large-scale image dataset containing pairs of images and their corresponding object-removed versions. Using these pairs, we train a diffusion model to inverse the inpainting process, effectively adding objects into images. Unlike other editing datasets, ours features natural target images instead of synthetic ones while ensuring source-target consistency by construction. Additionally, we utilize a large Vision-Language Model to provide detailed descriptions of the removed objects and a Large Language Model to convert these descriptions into diverse, natural-language instructions. Our quantitative and qualitative results show that the trained model surpasses existing models in both object addition and general editing tasks. Visit our project page for the released dataset and trained models.

Rerankers play a critical role in multimodal Retrieval-Augmented Generation (RAG) by refining ranking of an initial set of retrieved documents.Rerankers are typically trained using hard negative mining, whose goal is to select pages for each query which rank high, but are actually irrelevant.However, this selection process is typically passive and restricted to what the retriever can find in the available corpus, leading to several inherent limitations.These include: limited diversity, negative examples which are often not hard enough, low controllability, and frequent false negatives which harm training.Our paper proposes an alternative approach: Single-Page Hard Negative Query Generation, which goes the other way around.Instead of retrieving negative pages per query, we generate hard negative queries per page.Using an automated LLM-VLM pipeline, and given a page and its positive query, we create hard negatives by rephrasing the query to be as similar as possible in form and context, yet not answerable from the page.This paradigm enables fine-grained control over the generated queries, resulting in diverse, hard, and targeted negatives.It also supports efficient false negative verification.Our experiments show that rerankers trained with data generated using our approach outperform existing models and significantly improve retrieval performance 1 .