Multi-modal Relational Item Representation Learning for Inferring Substitutable and Complementary Items
sigir26-160 · Full research paper · Junting Wang, Chenghuan Guo, Yang Jiao, Yanhui Guo, Hari Sundaram, Yan Gao
Multimodal Denoising and Robustness · 50% Multimodal Retrieval and Embedding · 50%
Abstract
We study the problem of inferring substitutable and complementary items, which underpins applications such as alternative and follow-up purchase suggestions. Existing approaches typically learn from behavior-derived item-item associations using GNNs or leverage item content alone. However, these methods often overlook two key challenges: (i) user behaviors (e.g., co-view/co-purchase) only provide noisy weak supervision, and (ii) behavior signals are long-tailed, leaving many items with sparse associations. We propose MMSC, a self-supervised multi-modal relational representation learning framework that combines a multi-modal foundation model adapted to encode item metadata and a self-supervised denoising module that learns relationship-aware representations from noisy user behaviors, unified by a hierarchical aggregation mechanism. We further use LLM-assisted supervision to mitigate noise in behavior-derived supervision during training. Experiments on five real-world datasets show that MMSC consistently outperforms existing baselines by 26.1% for substitutable and 39.2% for complementary item inference, while remaining effective for cold-start items.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Minghui Fang 0002 | 67 | 71 OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup (2025) Multimodal Retrieval and Embedding 71 SyncTalklip: Highly Synchronized Lip-Readable Speaker Generation with Multi-Task Learning (2024) Multimodal Retrieval and Embedding 71 WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling (2025) Multimodal Retrieval and Embedding |
|
| 2 | Fengling Li 0001 | 35 | 49 SOIL: Contrastive Second-Order Interest Learning for Multimodal Recommendation (2024) Graph Contrastive and Self-Supervised Learning 49 Flip is Better than Noise: Unbiased Interest Generation for Multimedia Recommendation (2025) Diffusion and Flow Recommenders 36 Task-Adversarial Adaptation for Multi-modal Recommendation (2023) Domain Adaptation and Disentangled Representation for CDR |
Area chair
| Area chair | Match | |
|---|---|---|
| Xiaoye Qu | 39 |
Area-chair candidates (3)
| Xiaoye Qu assigned | 39 | 15 papers | |
| Xiaoyi Zeng | 32 | 11 papers | |
| Weiming Liu 0005 | 31 | 8 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Minghui Fang 0002 assigned | 67 | 3 | 71 OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup (2025) Multimodal Retrieval and Embedding 71 SyncTalklip: Highly Synchronized Lip-Readable Speaker Generation with Multi-Task Learning (2024) Multimodal Retrieval and Embedding |
|
| 2 | Haiyang Xu 0001 | 47 | 3 | 71 Efficient and Effective In-context Demonstration Selection with Coreset (2026) Multimodal Retrieval and Embedding 71 Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation (2023) Multimodal Retrieval and Embedding |
|
| 3 | Haokun Wen | 45 | 3 | 57 Differential-Perceptive and Retrieval-Augmented MLLM for Change Captioning (2024) Multimodal and Graph RAG 46 Multimodal Compatibility Modeling via Exploring the Consistent and Complementary Correlations (2021) Universal Multimodal Embedding and MLLM Retrieval |
|
| 4 | Yinwei Wei | 42 | 3 | 65 Leveraging Multimodal Features and Item-level User Feedback for Bundle Construction (2024) Graph Contrastive and Self-Supervised Learning 65 Breaking Isolation: Multimodal Graph Fusion for Multimedia Recommendation by Edge-wise Modulation (2022) Graph Contrastive and Self-Supervised Learning |
|
| 5 | Jian Lang | 41 | 3 | 71 REDEEMing Modality Information Loss: Retrieval-Guided Conditional Generation for Severely Modality Missing Learning (2025) Universal Multimodal Embedding and MLLM Retrieval 46 Seeing the Unseen in Micro-Video Popularity Prediction: Self-Correlation Retrieval for Missing Modality Generation (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 6 | Jinpeng Wang 0002 | 40 | 3 | 71 Imagine with Layout and Sketch: Enhancing Vision-Language Retrieval with Dual-Stream Multi-Modal Query Refinement (2026) Universal Multimodal Embedding and MLLM Retrieval 71 Suit the Remedy to the Retriever: Interpretable Query Optimization with Retriever Preference Alignment for Vision-Language Retrieval (2026) Universal Multimodal Embedding and MLLM Retrieval |
|
| 7 | Zihao Yue | 40 | 3 | 44 Learning Descriptive Image Captioning via Semipermeable Maximum Likelihood Estimation (2023) Multimodal Evaluation and Benchmarks 44 POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering (2026) Multimodal Evaluation and Benchmarks |
|
| 9 | Yanzhao Zhang | 39 | 3 | 71 Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs (2025) Universal Multimodal Embedding and MLLM Retrieval 71 Bridging Modalities: Improving Universal Multimodal Retrieval by Multimodal Large Language Models (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 10 | Hao Deng 0011 | 39 | 3 | 61 ESANS: Effective and Semantic-Aware Negative Sampling for Large-Scale Retrieval Systems (2025) Multimodal Denoising and Robustness 39 MMQ: Multimodal Mixture-of-Quantization Tokenization for Semantic ID Generation and User Behavioral Adaptation (2026) Item Tokenization and Semantic IDs |
|
| 11 | Xin Zhang 0091 | 37 | 3 | 71 Multi-Modal Correction Network for Recommendation (2025) Multimodal Denoising and Robustness 29 Deep Dynamic Interest Learning With Session Local and Global Consistency for Click-Through Rate Predictions (2022) CTR and Conversion Rate Prediction |
|
| 12 | Linchao Zhu | 36 | 3 | 58 Attention as Selector: Unlocking VLM Attention for Long Document Page Retrieval (2026) Universal Multimodal Embedding and MLLM Retrieval 46 Knowledge-Enhanced Dual-Stream Zero-Shot Composed Image Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 13 | Sheng-Chieh Lin | 36 | 3 | 71 Mm-Embed: Universal Multimodal Retrieval with Multimodal LLMS (2025) Universal Multimodal Embedding and MLLM Retrieval 58 Unifying Multimodal Retrieval via Document Screenshot Embedding (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 14 | Jiuqiang Li 0002 | 36 | 3 | 60 Multi-view Semantic Contrastive Alignment for Multimodal Recommendation (2026) Universal Multimodal Embedding and MLLM Retrieval 25 Disentangled Contrastive Bundle Recommendation with Conditional Diffusion (2025) Diffusion and Flow Recommenders |
|
| 15 | Fengling Li 0001 assigned | 35 | 3 | 49 SOIL: Contrastive Second-Order Interest Learning for Multimodal Recommendation (2024) Graph Contrastive and Self-Supervised Learning 49 Flip is Better than Noise: Unbiased Interest Generation for Multimedia Recommendation (2025) Diffusion and Flow Recommenders |
|
| 16 | Zheng Zhang 0006 | 35 | 3 | 35 From Sparse to Dense: Semantic Graph Evolutionary Hashing for Unsupervised Cross-Modal Retrieval (2022) Cross-Modal Hashing and Quantization 35 Work Together: Correlation-Identity Reconstruction Hashing for Unsupervised Cross-Modal Retrieval (2023) Cross-Modal Hashing and Quantization |
|
| 17 | Liwei Wang 0009 | 35 | 3 | 46 Learning Deep Structure-Preserving Image-Text Embeddings (2016) Universal Multimodal Embedding and MLLM Retrieval 46 Learning Two-Branch Neural Networks for Image-Text Matching Tasks (2019) Universal Multimodal Embedding and MLLM Retrieval |
|
| 18 | Meiyu Liang | 34 | 3 | 46 Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 46 Knowledge Graph Enhanced Multimodal Transformer for Image-Text Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 19 | Junchen Fu | 34 | 3 | 56 Teach Me How to Denoise: A Universal Framework for Denoising Multi-modal Recommender Systems via Guided Calibration (2025) Multimodal Generative Recommendation 44 Benchmarking Multimodal Large Language Models for missing modality completion in product catalogues (2026) Multimodal Evaluation and Benchmarks |
|
| 20 | Yongxin Ni | 34 | 3 | 56 Teach Me How to Denoise: A Universal Framework for Denoising Multi-modal Recommender Systems via Guided Calibration (2025) Multimodal Generative Recommendation 42 Causality-Inspired Fair Representation Learning for Multimodal Recommendation (2025) Bias Mitigation Methods |
|
| 21 | Archiki Prasad | 34 | 3 | 71 How Accents Confound: Probing for Accent Information in End-to-End Speech Recognition Systems (2020) Multimodal Retrieval and Embedding 71 An Investigation of End-to-End Models for Robust Speech Recognition (2021) Multimodal Retrieval and Embedding |
|
| 22 | Qijiong Liu | 33 | 3 | 44 EasyGen: Easing Multimodal Generation with BiDiffuser and LLMs (2024) Multimodal Evaluation and Benchmarks 43 Multimodal Pretraining, Adaptation, and Generation for Recommendation: A Survey (2024) Multimodal Generative Recommendation |
|
| 23 | Riwei Lai | 32 | 3 | 71 WISE: Word-Level Interaction-Based Multimodal Fusion for Speech Emotion Recognition (2020) Multimodal Retrieval and Embedding 31 Disentangled Negative Sampling for Collaborative Filtering (2023) Graph Contrastive and Self-Supervised Learning |
|
| 24 | Ruqing Zhang 0001 | 32 | 3 | 37 B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders 37 PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders |
|
| 25 | Navve Wasserman | 32 | 3 | 40 DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025) Universal Multimodal Embedding and MLLM Retrieval 40 REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025) Universal Multimodal Embedding and MLLM Retrieval |
|
| 27 | Weipeng Chen | 32 | 3 | 44 AITQE: An Adaptive Image-Text Quality Enhancer for Scalable MLLM Pretraining (2026) Multimodal Evaluation and Benchmarks 40 Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs (2025) Video and Temporal Understanding |
|
| 28 | Di Wu 0054 | 32 | 3 | 45 VisRet: Visualization Improves Knowledge-Intensive Text-to-Image Retrieval (2026) Multimodal and Graph RAG 34 The Factuality Tax of Diversity-Intervened Text-to-Image Generation: Benchmark and Fact-Augmented Intervention (2024) Multimodal Evaluation and Benchmarks |
|
| 29 | Huiqiang Jiang | 31 | 3 | 46 Attentive Mask CLIP (2023) Universal Multimodal Embedding and MLLM Retrieval 36 Multi-Level Knowledge Distillation for Out-of-Distribution Detection in Text (2023) Text Embedding and Representation Learning |
|
| 30 | Qingyi Si | 31 | 3 | 44 Multimodal Table Understanding (2024) Multimodal Evaluation and Benchmarks 44 Towards Flexible Evaluation for Generative Visual Question Answering (2024) Multimodal Evaluation and Benchmarks |
|
| 31 | Yi Zhang 0103 | 31 | 3 | 50 Intent Representation Learning with Large Language Model for Recommendation (2025) Collaborative-Semantic Representation Alignment 33 MixRec: Individual and Collective Mixing Empowers Data Augmentation for Recommender Systems (2025) Graph Contrastive and Self-Supervised Learning |
|
| 32 | Nancy F. Chen | 31 | 3 | 71 Rescoring Hypothesized Detections of Out-of-Vocabulary Keywords Using Subword Samples (2016) Multimodal Retrieval and Embedding 44 Controllable Neural Dialogue Summarization with Personal Named Entity Planning (2021) Multimodal Evaluation and Benchmarks |