MECI: Multi-Element Collaborative Interaction for Multimodal Entity Linking
sigir26-059 · Full research paper · Jie Peng, Yongxue Shan, Yongfu Zha, Xiaodong Wang
Entity Linking and Alignment · 50% Universal Multimodal Embedding and M · 50%
Abstract
Multimodal Entity Linking (MEL) aims to disambiguate mentions in multimodal contexts by grounding them to specific entities in a knowledge base. A pivotal challenge in MEL is capturing multi-level correspondences: the semantic consistency between mention-entity pairs and the complementary correlations across modalities. However, existing methods often suffer from element dominance due to their reliance on coupled interactions or coarse global aggregations. In response, we propose the Multi-Element Collaborative Interaction (MECI) framework. First, to capture multi-element mention-entity correspondences, we develop a Multi-view Experts Network that leverages a ''divide-and-conquer'' strategy for decoupled feature learning to mitigate element dominance, supported by a KL-guided routing mechanism that governs expert specialization and collaboration. Furthermore, to model cross-modal complementary correlations, we propose a Hierarchical Multimodal Interaction Module, where a dynamic modality-aware weighting network refines interactions across hierarchical semantic levels, thereby integrating multi-granular evidence to counteract element dominance. Finally, we incorporate a generative semantic refinement stage that utilizes large language models for zero-shot re-ranking. Extensive experiments on WikiDiverse, RichpediaMEL, and WikiMEL show that MECI consistently outperforms state-of-the-art baselines, improving Hits@1 by 1.95%, 7.30%, and 2.31%, respectively.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Xuhui Sui | 63 | 64 ZS-MNET: A zero-shot learning based approach to multimodal named entity typing (2025) Universal Multimodal Embedding and MLLM Retrieval 64 MCIL: Multimodal Counterfactual Instance Learning for Low-resource Entity-based Multimodal Information Extraction (2024) Universal Multimodal Embedding and MLLM Retrieval 64 Multimodal Graph-Based Variational Mixture of Experts Network for Zero-Shot Multimodal Information Extraction (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 2 | Xinying Qian | 54 | 63 SMIR: Span-based multi-grained information refinement for joint multimodal entity-relation extraction (2026) Universal Multimodal Embedding and MLLM Retrieval 63 Hyperbolic Multimodal Generative Representation Learning for Generalized Zero-Shot Multimodal Information Extraction (2026) Universal Multimodal Embedding and MLLM Retrieval 39 TimeR⁴ : Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering (2024) Multi-Hop and Iterative Retrieval Reasoning |
Area chair
| Area chair | Match | |
|---|---|---|
| Hao Fei 0001 | 35 |
Area-chair candidates (3)
| Xiaoye Qu | 38 | 15 papers | |
| Chenyan Xiong | 37 | 15 papers | |
| Hao Fei 0001 assigned | 35 | 4 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Xuhui Sui assigned | 63 | 3 | 64 ZS-MNET: A zero-shot learning based approach to multimodal named entity typing (2025) Universal Multimodal Embedding and MLLM Retrieval 64 MCIL: Multimodal Counterfactual Instance Learning for Low-resource Entity-based Multimodal Information Extraction (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 2 | Haokun Wen | 63 | 3 | 69 Personalized Fashion Compatibility Modeling via Metapath-guided Heterogeneous Graph Learning (2022) Universal Multimodal Embedding and MLLM Retrieval 63 ENCODER: Entity Mining and Modification Relation Binding for Composed Image Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 3 | Liwei Wang 0009 | 57 | 3 | 63 Learning Deep Structure-Preserving Image-Text Embeddings (2016) Universal Multimodal Embedding and MLLM Retrieval 63 A Fast and Accurate One-Stage Approach to Visual Grounding (2019) Universal Multimodal Embedding and MLLM Retrieval |
|
| 4 | Jian Lang | 56 | 3 | 63 Generative Thinking, Corrective Action: User-Friendly Composed Image Retrieval via Automatic Multi-Agent Collaboration (2025) Universal Multimodal Embedding and MLLM Retrieval 63 Retrieval-Augmented Dynamic Prompt Tuning for Incomplete Multimodal Learning (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 5 | Yuzhong Qu | 56 | 3 | 63 Summarizing Entity Descriptions for Effective and Efficient Human-centered Entity Linking (2015) Entity Linking and Alignment 63 Revisiting Embedding-Based Entity Alignment: A Robust and Adaptive Method (2023) Entity Linking and Alignment |
|
| 6 | Xinying Qian assigned | 54 | 3 | 63 SMIR: Span-based multi-grained information refinement for joint multimodal entity-relation extraction (2026) Universal Multimodal Embedding and MLLM Retrieval 63 Hyperbolic Multimodal Generative Representation Learning for Generalized Zero-Shot Multimodal Information Extraction (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 7 | Navve Wasserman | 53 | 3 | 69 DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025) Universal Multimodal Embedding and MLLM Retrieval 69 REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 8 | Minghui Fang 0002 | 50 | 3 | 63 CART: A Generative Cross-Modal Retrieval Framework With Coarse-To-Fine Semantic Modeling (2025) Universal Multimodal Embedding and MLLM Retrieval 63 Open-Set Cross Modal Generalization via Multimodal Unified Representation (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 9 | Yanzhao Zhang | 47 | 3 | 63 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval 63 Towards Text-Image Interleaved Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 10 | Junjie Zhou 0001 | 46 | 3 | 63 VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval 63 MegaPairs: Massive Data Synthesis for Universal Multimodal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 11 | Fu Zhang 0001 | 45 | 3 | 63 Probing Relative Interaction and Dynamic Calibration in Multi-modal Entity Alignment (2025) Entity Linking and Alignment 63 SGMEA: Structure-Guided Multimodal Entity Alignment (2025) Entity Linking and Alignment |
|
| 12 | Wonduk Seo | 45 | 3 | 63 Question-to-Knowledge (Q2K): Multi-Agent Generation of Inspectable Facts for Product Mapping (2025) Entity Linking and Alignment 35 A New Query Expansion Approach for Enhancing Information Retrieval via Agent-Mediated Dialogic Inquiry (2026) Query Reformulation and Expansion |
|
| 13 | Jinpeng Wang 0002 | 43 | 3 | 63 Modeling Uncertainty in Composed Image Retrieval via Probabilistic Embeddings (2025) Universal Multimodal Embedding and MLLM Retrieval 63 Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learning (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 14 | Jingwei Cheng | 43 | 3 | 63 Probing Relative Interaction and Dynamic Calibration in Multi-modal Entity Alignment (2025) Entity Linking and Alignment 63 SGMEA: Structure-Guided Multimodal Entity Alignment (2025) Entity Linking and Alignment |
|
| 15 | Linchao Zhu | 42 | 3 | 64 Attention as Selector: Unlocking VLM Attention for Long Document Page Retrieval (2026) Universal Multimodal Embedding and MLLM Retrieval 63 Knowledge-Enhanced Dual-Stream Zero-Shot Composed Image Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 16 | Meiyu Liang | 41 | 3 | 63 Knowledge Graph Enhanced Multimodal Transformer for Image-Text Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval 63 Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 17 | Qinggang Zhang | 41 | 3 | 63 Entity Alignment with Noisy Annotations from Large Language Models (2024) Entity Linking and Alignment 57 LoSemB: Logic-Guided Semantic Bridging for Inductive Tool Retrieval (2026) Reasoning Evaluation and Benchmarks |
|
| 18 | Xiangyu Ke | 39 | 3 | 63 In-context Clustering-based Entity Resolution with Large Language Models: A Design Space Exploration (2025) Entity Linking and Alignment 63 Select Your Questions Wisely: For Entity Resolution With Crowd Errors (2017) Entity Linking and Alignment |
|
| 19 | Jianfeng Dong | 39 | 3 | 63 Feature Re-Learning with Data Augmentation for Content-based Video Recommendation (2018) Universal Multimodal Embedding and MLLM Retrieval 61 Exploring Content-based Video Relevance for Video Click-Through Rate Prediction (2019) Universal Multimodal Embedding and MLLM Retrieval |
|
| 20 | Sheng-Chieh Lin | 39 | 3 | 64 Unifying Multimodal Retrieval via Document Screenshot Embedding (2024) Universal Multimodal Embedding and MLLM Retrieval 63 Mm-Embed: Universal Multimodal Retrieval with Multimodal LLMS (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 22 | Di Wu 0054 | 38 | 3 | 70 VisRet: Visualization Improves Knowledge-Intensive Text-to-Image Retrieval (2026) Multimodal and Graph RAG 34 LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (2025) Conversational Search and Retrieval |
|
| 23 | Tingyu Song | 38 | 3 | 64 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval 38 VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videos (2025) Video and Temporal Understanding |
|
| 24 | Haiyang Xu 0001 | 37 | 3 | 39 Efficient and Effective In-context Demonstration Selection with Coreset (2026) Multimodal Retrieval and Embedding 39 Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation (2023) Multimodal Retrieval and Embedding |
|
| 26 | Jiuqiang Li 0002 | 37 | 3 | 63 Multi-view Semantic Contrastive Alignment for Multimodal Recommendation (2026) Universal Multimodal Embedding and MLLM Retrieval 22 Graph Diffusive Self-Supervised Learning for Social Recommendation (2024) Social and Group Recommendation |
|
| 27 | Zihao Yue | 36 | 3 | 38 Learning Descriptive Image Captioning via Semipermeable Maximum Likelihood Estimation (2023) Multimodal Evaluation and Benchmarks 38 POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering (2026) Multimodal Evaluation and Benchmarks |
|
| 28 | Jinheon Baek | 36 | 3 | 70 UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities (2026) Multimodal and Graph RAG 38 An Empirical Study of Multilingual Reasoning Distillation for Question Answering (2024) Multi-hop and Complex Reasoning QA |
|
| 30 | Huiqiang Jiang | 35 | 3 | 63 Attentive Mask CLIP (2023) Universal Multimodal Embedding and MLLM Retrieval 36 SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents (2025) Conversational Search and Retrieval |
|
| 31 | Xi Ye 0003 | 35 | 3 | 36 RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering (2022) Knowledge Graph Question Answering 36 Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking (2025) Multi-hop and Complex Reasoning QA |
|
| 32 | Jing Liu 0022 | 35 | 3 | 64 A Hierarchical Entity-Based Approach to Structuralize User Generated Content in Social Media: A Case of Yahoo! Answers (2013) Domain-Specific and Structured Retrieval 35 RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking (2021) Dense Bi-Encoder Training and Representation Learning |
|
| 33 | Mariano Barone | 35 | 3 | 39 A Multilingual Multimodal Medical Examination Dataset for Visual Question Answering in Healthcare (2025) Retrieval-Augmented and Open-Domain QA 38 AMICO: A Semantic and Multimodal Framework for AI-Assisted Clinical Reporting (2025) Multimodal Evaluation and Benchmarks |