Unsupervised 2D Image-Based 3D Model Retrieval via Decision Boundary Alignment and Graph Semantic Propagation
sigir26-096 · Full research paper · Nian Hu, Yibo Zhao, Xinhui Li, Chen Li, Cong Liu, Zan Gao
Video and Moment Retrieval · 100%
Abstract
Unsupervised 2D image-based 3D model retrieval (IBMR) aims to retrieve semantically relevant 3D shapes for a given 2D image query when 3D annotations are unavailable. This setting is challenging due to severe modality gaps, category-imbalanced mini-batches, inconsistent cross-domain decision boundaries, and mismatched semantic neighborhood structures. In this paper, we propose a unified framework that integrates Category-Aligned Sampling (CAS), Decision Boundary Alignment (DBA), and Graph Semantic Propagation (GSP) into a single optimization paradigm. CAS constructs category-consistent mini-batches to stabilize crossmodal learning. Built upon CAS, DBA leverages a masked Margin Disparity Discrepancy to regularize cross-domain class decision boundaries via an adversarial min-max objective, encouraging discriminative separation beyond marginal feature matching. To complement boundary-level regularization, GSP builds a crossdomain affinity graph over 2D and 3D samples and propagates supervision-induced relational structure through semantic message passing, explicitly preserving instance-level neighborhood consistency that is critical for retrieval. Extensive experiments on MI3DOR and MI3DOR-2 demonstrate consistent improvements over representative unsupervised IBMR baselines.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Jianfeng Dong | 71 | 100 Dual Encoding for Zero-Example Video Retrieval (2019) Video and Moment Retrieval 100 Video Moment Retrieval With Cross-Modal Neural Architecture Search (2022) Video and Moment Retrieval 100 FeatInter: Exploring fine-grained object features for video-text retrieval (2022) Video and Moment Retrieval |
|
| 2 | Zhe Xu 0009 | 46 | 100 Point-Supervised Video Temporal Grounding (2023) Video and Moment Retrieval 100 Exploiting Intrinsic Multilateral Logical Rules for Weakly Supervised Natural Language Video Localization (2024) Video and Moment Retrieval 100 HiSA: Hierarchically Semantic Associating for Video Temporal Grounding (2022) Video and Moment Retrieval |
Area chair
| Area chair | Match | |
|---|---|---|
| Xun Yang 0001 | 50 |
Area-chair candidates (5)
| Xun Yang 0001 assigned | 50 | 4 papers | |
| Xiaoye Qu | 49 | 15 papers | |
| Hongjin Qian | 33 | 12 papers | |
| Chenyan Xiong | 33 | 15 papers | |
| Kam-Fai Wong | 32 | 6 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Jianfeng Dong assigned | 71 | 3 | 100 Dual Encoding for Zero-Example Video Retrieval (2019) Video and Moment Retrieval 100 Video Moment Retrieval With Cross-Modal Neural Architecture Search (2022) Video and Moment Retrieval |
|
| 2 | Linchao Zhu | 64 | 3 | 100 Align and Tell: Boosting Text-Video Retrieval With Local Alignment and Fine-Grained Supervision (2023) Video and Moment Retrieval 100 Stitching Segments and Sentences towards Generalization in Video-Text Pre-training (2024) Video and Moment Retrieval |
|
| 5 | Huaiwen Zhang | 47 | 3 | 100 Sequence-Event Semantic Consistent Learning for Text-to-Motion Retrieval (2025) Video and Moment Retrieval 100 Robust Video-Text Retrieval Via Noisy Pair Calibration (2023) Video and Moment Retrieval |
|
| 6 | Zhe Xu 0009 assigned | 46 | 3 | 100 Point-Supervised Video Temporal Grounding (2023) Video and Moment Retrieval 100 Exploiting Intrinsic Multilateral Logical Rules for Weakly Supervised Natural Language Video Localization (2024) Video and Moment Retrieval |
|
| 7 | Haokun Wen | 41 | 3 | 45 A Comprehensive Survey on Composed Image Retrieval (2026) Universal Multimodal Embedding and MLLM Retrieval 45 Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 8 | Haiyang Xu 0001 | 41 | 3 | 100 Learning Semantics-Grounded Vocabulary Representation for Video-Text Retrieval (2023) Video and Moment Retrieval 100 Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval (2024) Video and Moment Retrieval |
|
| 9 | Junjie Zhou 0001 | 39 | 3 | 100 MomentSeeker: A Task-Oriented Benchmark For Long-Video Moment Retrieval (2025) Video and Moment Retrieval 46 Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding (2025) Video and Temporal Understanding |
|
| 10 | Jinpeng Wang 0002 | 37 | 3 | 100 Enhancing Partially Relevant Video Retrieval with Hyperbolic Learning (2025) Video and Moment Retrieval 45 Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learning (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 11 | Jian Lang | 37 | 3 | 45 Seeing the Unseen in Micro-Video Popularity Prediction: Self-Correlation Retrieval for Missing Modality Generation (2025) Universal Multimodal Embedding and MLLM Retrieval 45 Generative Thinking, Corrective Action: User-Friendly Composed Image Retrieval via Automatic Multi-Agent Collaboration (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 12 | Zheng Zhang 0006 | 37 | 3 | 37 Cross-Domain Transfer Hashing for Efficient Cross-Modal Retrieval (2024) Cross-Modal Hashing and Quantization 37 Contrastive Incomplete Cross-Modal Hashing (2024) Cross-Modal Hashing and Quantization |
|
| 13 | Minghui Fang 0002 | 36 | 3 | 45 Open-Set Cross Modal Generalization via Multimodal Unified Representation (2025) Universal Multimodal Embedding and MLLM Retrieval 45 Multimodal Conditional Retrieval with High Controllability (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 14 | Liwei Wang 0009 | 36 | 3 | 46 AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning (2025) Video and Temporal Understanding 45 Learning Two-Branch Neural Networks for Image-Text Matching Tasks (2019) Universal Multimodal Embedding and MLLM Retrieval |
|
| 15 | Sheng-Chieh Lin | 35 | 3 | 42 Contextualized Query Embeddings for Conversational Search (2021) Conversational Search and Retrieval 41 Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting (2021) Conversational Search and Retrieval |
|
| 16 | Meiyu Liang | 35 | 3 | 45 Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 45 Knowledge Graph Enhanced Multimodal Transformer for Image-Text Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 17 | Navve Wasserman | 34 | 3 | 40 REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025) Universal Multimodal Embedding and MLLM Retrieval 40 DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 18 | Zihao Yue | 34 | 3 | 39 OpenMMEgo: Enhancing Egocentric Understanding for LMMs with Open Weights and Data (2025) Video and Temporal Understanding 39 Movie101v2: Improved Movie Narration Benchmark (2025) Video and Temporal Understanding |
|
| 21 | Fengling Li 0001 | 33 | 3 | 37 Cross-Domain Transfer Hashing for Efficient Cross-Modal Retrieval (2024) Cross-Modal Hashing and Quantization 37 Noise-Robust Generative Hashing for Cross-Modal Retrieval (2026) Cross-Modal Hashing and Quantization |
|
| 22 | Xueguang Ma | 33 | 3 | 38 PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval (2024) Text Embedding and Representation Learning 36 R2LLMs: Retrieval and Ranking with LLMs (2025) Reasoning-Intensive Retrieval and Ranking |
|
| 23 | Xiao Wang 0043 | 33 | 3 | 37 Improving zero-shot retrieval using dense external expansion (2022) Query Reformulation and Expansion 34 Neural Pseudo-Relevance Feedback Models for Sparse and Dense Retrieval (2022) Query Reformulation and Expansion |
|
| 24 | Yanzhao Zhang | 33 | 3 | 45 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval 45 Towards Text-Image Interleaved Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 25 | Shengyao Zhuang | 33 | 3 | 47 Embark on DenseQuest: A System for Selecting the Best Dense Retriever for a Custom Collection (2024) Zero-Shot and Domain Generalization for Retrieval 37 Leveraging LLMs for Unsupervised Dense Retriever Ranking (2024) Query Performance Prediction |
|
| 26 | Huiqiang Jiang | 32 | 3 | 45 Attentive Mask CLIP (2023) Universal Multimodal Embedding and MLLM Retrieval 45 SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents (2025) Conversational Search and Retrieval |
|
| 27 | Ruqing Zhang 0001 | 32 | 3 | 35 Generative Retrieval Meets Multi-Graded Relevance (2024) Generative Retrieval 35 Continual Learning for Generative Retrieval over Dynamic Corpora (2023) Generative Retrieval |
|
| 29 | Xinyu Ma 0001 | 31 | 3 | 37 Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense Retrieval (2023) Sparse and Dense Retrieval Encoders 35 Replication and Exploration of Generative Retrieval over Dynamic Corpora (2025) Generative Retrieval |
|
| 30 | Antonio Mallia | 31 | 3 | 34 Improved Learned Sparse Retrieval with Corpus-Specific Vocabularies (2024) Learned Sparse Retrieval Efficiency 34 Learning Passage Impacts for Inverted Indexes (2021) Learned Sparse Retrieval Efficiency |
|
| 31 | Xinying Qian | 31 | 3 | 45 Hyperbolic Multimodal Generative Representation Learning for Generalized Zero-Shot Multimodal Information Extraction (2026) Universal Multimodal Embedding and MLLM Retrieval 45 SMIR: Span-based multi-grained information refinement for joint multimodal entity-relation extraction (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 32 | Tingyu Song | 31 | 3 | 40 Rethinking Composed Image Retrieval Evaluation: A Fine-Grained Benchmark from Image Editing (2026) Universal Multimodal Embedding and MLLM Retrieval 39 VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videos (2025) Video and Temporal Understanding |
|
| 33 | Negar Arabzadeh | 30 | 3 | 34 Enhanced Retrieval Effectiveness through Selective Query Generation (2024) Query Reformulation and Expansion 34 ReFormeR: Learning and Applying Explicit Query Reformulation Patterns (2026) Query Reformulation and Expansion |
|
| 34 | Jingtao Zhan | 30 | 3 | 42 Combining Multiple Supervision for Robust Zero-Shot Dense Retrieval (2024) Sparse and Dense Retrieval Encoders 35 Scaling Laws For Dense Retrieval (2024) Sparse and Dense Retrieval Encoders |
|
| 35 | Peitian Zhang | 30 | 3 | 46 Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding (2025) Video and Temporal Understanding 36 INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning (2024) LLM-Based Reranking |