CoMCo: Consistency-Aware Multi-Agent Coordination for Zero-Shot Cross-Modal Entity Matching
Abstract
Entity Matching (EM) is a fundamental task in data integration, traditionally studied over structured data such as tables and knowledge graphs. In modern repositories, real-world objects are often represented across multiple modalities, including structured entities with symbolic attributes and visual entities in image-centric collections. This motivates cross-modal entity matching, which aims to identify visual-structured entity pairs that refer to the same object. Existing methods typically rely on pretrained vision-language models to compute entity pair similarity and derive correspondence via local ranking, which, however, can be unreliable given noisy and ambiguous cross-modal data and may produce globally inconsistent correspondences across related entities. Multimodal large language models (MLLMs) offer richer cross-modal cues for matching, but exhaustive MLLM reasoning over large candidate spaces is prohibitively expensive. To address these limitations, in this work, we propose øurq, a blackboard-based multi-agent framework for zero-shot cross-modal entity matching that performs iterative, self-correcting refinement by fusing multiple matching signals, explicitly regulating global consistency, and selectively invoking an MLLM only for hard cases. We further construct two new benchmarks from real-world visual and structured data. Extensive experiments show that øurq consistently outperforms competitive baselines, providing an effective solution for zero-shot cross-modal entity matching. We release our data and code at https://github.com/Q-17/CoMCo.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Xuhui Sui | 63 | 64 MCIL: Multimodal Counterfactual Instance Learning for Low-resource Entity-based Multimodal Information Extraction (2024) Universal Multimodal Embedding and MLLM Retrieval 64 Multimodal Graph-Based Variational Mixture of Experts Network for Zero-Shot Multimodal Information Extraction (2025) Universal Multimodal Embedding and MLLM Retrieval 64 Hyperbolic Multimodal Generative Representation Learning for Generalized Zero-Shot Multimodal Information Extraction (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 2 | Junjie Zhou 0001 | 46 | 63 MegaPairs: Massive Data Synthesis for Universal Multimodal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 63 VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval 63 Any Information Is Just Worth One Single Screenshot: Unifying Search With Visualized Information Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
Area chair
| Area chair | Match | |
|---|---|---|
| Hao Fei 0001 | 35 |
Area-chair candidates (3)
| Xiaoye Qu | 38 | 15 papers | |
| Chenyan Xiong | 37 | 15 papers | |
| Hao Fei 0001 assigned | 35 | 4 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Xuhui Sui assigned | 63 | 3 | 64 MCIL: Multimodal Counterfactual Instance Learning for Low-resource Entity-based Multimodal Information Extraction (2024) Universal Multimodal Embedding and MLLM Retrieval 64 Multimodal Graph-Based Variational Mixture of Experts Network for Zero-Shot Multimodal Information Extraction (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 2 | Haokun Wen | 63 | 3 | 69 Personalized Fashion Compatibility Modeling via Metapath-guided Heterogeneous Graph Learning (2022) Universal Multimodal Embedding and MLLM Retrieval 63 HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 3 | Liwei Wang 0009 | 57 | 3 | 63 Learning Two-Branch Neural Networks for Image-Text Matching Tasks (2019) Universal Multimodal Embedding and MLLM Retrieval 63 Learning Deep Structure-Preserving Image-Text Embeddings (2016) Universal Multimodal Embedding and MLLM Retrieval |
|
| 4 | Jian Lang | 56 | 3 | 63 Generative Thinking, Corrective Action: User-Friendly Composed Image Retrieval via Automatic Multi-Agent Collaboration (2025) Universal Multimodal Embedding and MLLM Retrieval 63 Echoes in the Feed: Evolution-aware Prompt-augmented Micro-video Popularity Prediction (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 5 | Yuzhong Qu | 56 | 3 | 63 Revisiting Embedding-Based Entity Alignment: A Robust and Adaptive Method (2023) Entity Linking and Alignment 63 Automated Comparative Table Generation for Facilitating Human Intervention in Multi-Entity Resolution (2018) Entity Linking and Alignment |
|
| 6 | Xinying Qian | 54 | 3 | 63 Hyperbolic Multimodal Generative Representation Learning for Generalized Zero-Shot Multimodal Information Extraction (2026) Universal Multimodal Embedding and MLLM Retrieval 63 SMIR: Span-based multi-grained information refinement for joint multimodal entity-relation extraction (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 7 | Navve Wasserman | 53 | 3 | 69 REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025) Universal Multimodal Embedding and MLLM Retrieval 69 DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 8 | Minghui Fang 0002 | 50 | 3 | 63 Open-Set Cross Modal Generalization via Multimodal Unified Representation (2025) Universal Multimodal Embedding and MLLM Retrieval 63 CART: A Generative Cross-Modal Retrieval Framework With Coarse-To-Fine Semantic Modeling (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 9 | Yanzhao Zhang | 47 | 3 | 63 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval 63 Towards Text-Image Interleaved Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 10 | Junjie Zhou 0001 assigned | 46 | 3 | 63 MegaPairs: Massive Data Synthesis for Universal Multimodal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 63 VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 11 | Fu Zhang 0001 | 45 | 3 | 63 Breaking the Noise Barrier: LLM-Guided Semantic Filtering and Enhancement for Multi-Modal Entity Alignment (2025) Entity Linking and Alignment 63 Probing Relative Interaction and Dynamic Calibration in Multi-modal Entity Alignment (2025) Entity Linking and Alignment |
|
| 12 | Wonduk Seo | 45 | 3 | 63 Question-to-Knowledge (Q2K): Multi-Agent Generation of Inspectable Facts for Product Mapping (2025) Entity Linking and Alignment 35 A New Query Expansion Approach for Enhancing Information Retrieval via Agent-Mediated Dialogic Inquiry (2026) Query Reformulation and Expansion |
|
| 13 | Jinpeng Wang 0002 | 43 | 3 | 63 Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learning (2026) Universal Multimodal Embedding and MLLM Retrieval 63 Modeling Uncertainty in Composed Image Retrieval via Probabilistic Embeddings (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 14 | Jingwei Cheng | 43 | 3 | 63 Breaking the Noise Barrier: LLM-Guided Semantic Filtering and Enhancement for Multi-Modal Entity Alignment (2025) Entity Linking and Alignment 63 Probing Relative Interaction and Dynamic Calibration in Multi-modal Entity Alignment (2025) Entity Linking and Alignment |
|
| 15 | Linchao Zhu | 42 | 3 | 64 Attention as Selector: Unlocking VLM Attention for Long Document Page Retrieval (2026) Universal Multimodal Embedding and MLLM Retrieval 63 Knowledge-Enhanced Dual-Stream Zero-Shot Composed Image Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 16 | Meiyu Liang | 41 | 3 | 63 Knowledge Graph Enhanced Multimodal Transformer for Image-Text Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval 63 Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 17 | Qinggang Zhang | 41 | 3 | 63 Entity Alignment with Noisy Annotations from Large Language Models (2024) Entity Linking and Alignment 57 LoSemB: Logic-Guided Semantic Bridging for Inductive Tool Retrieval (2026) Reasoning Evaluation and Benchmarks |
|
| 18 | Xiangyu Ke | 39 | 3 | 63 Select Your Questions Wisely: For Entity Resolution With Crowd Errors (2017) Entity Linking and Alignment 63 In-context Clustering-based Entity Resolution with Large Language Models: A Design Space Exploration (2025) Entity Linking and Alignment |
|
| 19 | Jianfeng Dong | 39 | 3 | 63 Feature Re-Learning with Data Augmentation for Content-based Video Recommendation (2018) Universal Multimodal Embedding and MLLM Retrieval 61 Exploring Content-based Video Relevance for Video Click-Through Rate Prediction (2019) Universal Multimodal Embedding and MLLM Retrieval |
|
| 20 | Sheng-Chieh Lin | 39 | 3 | 64 Unifying Multimodal Retrieval via Document Screenshot Embedding (2024) Universal Multimodal Embedding and MLLM Retrieval 63 Mm-Embed: Universal Multimodal Retrieval with Multimodal LLMS (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 22 | Di Wu 0054 | 38 | 3 | 70 VisRet: Visualization Improves Knowledge-Intensive Text-to-Image Retrieval (2026) Multimodal and Graph RAG 34 LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (2025) Conversational Search and Retrieval |
|
| 23 | Tingyu Song | 38 | 3 | 64 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval 38 VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videos (2025) Video and Temporal Understanding |
|
| 24 | Haiyang Xu 0001 | 37 | 3 | 39 Efficient and Effective In-context Demonstration Selection with Coreset (2026) Multimodal Retrieval and Embedding 39 Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation (2023) Multimodal Retrieval and Embedding |
|
| 26 | Jiuqiang Li 0002 | 37 | 3 | 63 Multi-view Semantic Contrastive Alignment for Multimodal Recommendation (2026) Universal Multimodal Embedding and MLLM Retrieval 22 Graph Diffusive Self-Supervised Learning for Social Recommendation (2024) Social and Group Recommendation |
|
| 27 | Zihao Yue | 36 | 3 | 38 POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering (2026) Multimodal Evaluation and Benchmarks 38 Learning Descriptive Image Captioning via Semipermeable Maximum Likelihood Estimation (2023) Multimodal Evaluation and Benchmarks |
|
| 28 | Jinheon Baek | 36 | 3 | 70 UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities (2026) Multimodal and Graph RAG 38 An Empirical Study of Multilingual Reasoning Distillation for Question Answering (2024) Multi-hop and Complex Reasoning QA |
|
| 30 | Huiqiang Jiang | 35 | 3 | 63 Attentive Mask CLIP (2023) Universal Multimodal Embedding and MLLM Retrieval 36 SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents (2025) Conversational Search and Retrieval |
|
| 31 | Xi Ye 0003 | 35 | 3 | 36 RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering (2022) Knowledge Graph Question Answering 36 Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking (2025) Multi-hop and Complex Reasoning QA |
|
| 32 | Jing Liu 0022 | 35 | 3 | 64 A Hierarchical Entity-Based Approach to Structuralize User Generated Content in Social Media: A Case of Yahoo! Answers (2013) Domain-Specific and Structured Retrieval 35 RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking (2021) Dense Bi-Encoder Training and Representation Learning |
|
| 33 | Mariano Barone | 35 | 3 | 39 A Multilingual Multimodal Medical Examination Dataset for Visual Question Answering in Healthcare (2025) Retrieval-Augmented and Open-Domain QA 38 AMICO: A Semantic and Multimodal Framework for AI-Assisted Clinical Reporting (2025) Multimodal Evaluation and Benchmarks |