Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps
Abstract
While advanced foundation models like ModernBERT significantly outperform older architectures in dense retrieval, they surprisingly lag behind the aging BERT-base baseline in learned sparse retrieval (LSR). We identify the root cause as the Vocabulary Gap : modern tokenizers utilize raw, case-sensitive vocabularies designed for lossless reconstruction, which map single semantic units to redundant surface forms, wasting model capacity on morphological noise and hindering lexical matching. We formalize this intuition through a theoretical framework, demonstrating that appropriate vocabulary coarse-graining can tighten the generalization bounds by reducing complexity of the hypothesis class, provided that semantic integrity is preserved. To resolve this, we propose Vocabulary Transfer (VT), a model-agnostic framework that migrates advanced encoders to sparse-friendly, normalized vocabularies with minimal computational cost. VT utilizes a novel Semantic Initialization via spatial topology to preserve geometric structure and an Activation Potential Calibration (APC) mechanism to align pre-trained manifolds with sparsity constraints, preventing the dead neuron and dense collapse observed in standard fine-tuning. Empirically, VT is universally effective: it enables ModernBERT to achieve state-of-the-art performance on the BEIR benchmark (52.4 nDCG, a +4.7 improvement), resuscitates failing models like RoBERTa-large, and generalizes seamlessly to inference-free architectures and specialized domains. These results confirm that the performance lag is not an architectural deficiency but a solvable vocabulary mismatch. We've released our code and models. https://anonymous.4open.science/r/vocab-transfer/. All details included.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Xinyu Ma 0001 | 53 | 80 Pre-train a Discriminative Text Encoder for Dense Retrieval via Contrastive Span Prediction (2022) Sparse and Dense Retrieval Encoders 80 A Contrastive Pre-training Approach to Discriminative Autoencoder for Dense Retrieval (2022) Sparse and Dense Retrieval Encoders 80 B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders |
|
| 2 | Jingtao Zhan | 51 | 80 Scaling Laws For Dense Retrieval (2024) Sparse and Dense Retrieval Encoders 77 Combining Multiple Supervision for Robust Zero-Shot Dense Retrieval (2024) Sparse and Dense Retrieval Encoders 71 An Analysis of BERT in Document Ranking (2020) Sparse and Dense Retrieval Encoders |
Area chair
| Area chair | Match | |
|---|---|---|
| Chenyan Xiong | 43 |
Area-chair candidates (2)
| Chenyan Xiong assigned | 43 | 15 papers | |
| Xiaoyi Zeng COI: co-author | 40 | 11 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Xueguang Ma | 57 | 3 | 77 LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculum (2026) Learned Sparse Retrieval Efficiency 76 Fine-Tuning LLaMA for Multi-Stage Text Retrieval (2024) Sparse and Dense Retrieval Encoders |
|
| 2 | Antonio Mallia | 55 | 3 | 77 Improved Learned Sparse Retrieval with Corpus-Specific Vocabularies (2024) Learned Sparse Retrieval Efficiency 77 Learning Passage Impacts for Inverted Indexes (2021) Learned Sparse Retrieval Efficiency |
|
| 3 | Franco Maria Nardini | 55 | 3 | 86 Expansion via Prediction of Importance with Contextualization (2020) Sparse and Dense Retrieval Encoders 63 Effective Inference-Free Retrieval for Learned Sparse Representations (2025) Learned Sparse Retrieval Efficiency |
|
| 4 | Xiao Wang 0043 | 54 | 3 | 63 Reproducibility, Replicability, and Insights into Dense Multi-Representation Retrieval Models: from ColBERT to Col (2023) Learned Sparse Retrieval Efficiency 59 An Inspection of the Reproducibility and Replicability of TCT-ColBERT (2022) Learned Sparse Retrieval Efficiency |
|
| 5 | Xinyu Ma 0001 assigned | 53 | 3 | 80 Pre-train a Discriminative Text Encoder for Dense Retrieval via Contrastive Span Prediction (2022) Sparse and Dense Retrieval Encoders 80 A Contrastive Pre-training Approach to Discriminative Autoencoder for Dense Retrieval (2022) Sparse and Dense Retrieval Encoders |
|
| 6 | Jingtao Zhan assigned | 51 | 3 | 80 Scaling Laws For Dense Retrieval (2024) Sparse and Dense Retrieval Encoders 77 Combining Multiple Supervision for Robust Zero-Shot Dense Retrieval (2024) Sparse and Dense Retrieval Encoders |
|
| 7 | Sheng-Chieh Lin | 51 | 3 | 71 mAggretriever: A Simple yet Effective Approach to Zero-Shot Multilingual Dense Retrieval (2023) Sparse and Dense Retrieval Encoders 67 Aggretriever: A Simple Approach to Aggregate Textual Representations for Robust Dense Passage Retrieval (2023) Dense Bi-Encoder Training and Representation Learning |
|
| 8 | Huiqiang Jiang | 50 | 3 | 57 RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval (2025) Learned Sparse Retrieval Efficiency 57 LeanK: Learnable K Cache Channel Pruning for Efficient Decoding (2025) Learned Sparse Retrieval Efficiency |
|
| 9 | Shengyao Zhuang | 48 | 3 | 65 PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval (2024) Text Embedding and Representation Learning 58 LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculum (2026) Learned Sparse Retrieval Efficiency |
|
| 10 | Sebastian Bruch 0001 | 47 | 3 | 63 Forward Index Compression for Learned Sparse Retrieval (2026) Learned Sparse Retrieval Efficiency 59 An Analysis of Fusion Functions for Hybrid Retrieval (2024) Dense Bi-Encoder Training and Representation Learning |
|
| 11 | Ruqing Zhang 0001 | 47 | 3 | 80 B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders 80 PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders |
|
| 12 | Zheng Zhang 0006 COI: co-author | 45 | 3 | 45 Task-adaptive Asymmetric Deep Cross-modal Hashing (2021) Cross-Modal Hashing and Quantization 45 Inductive Structure Consistent Hashing via Flexible Semantic Calibration (2021) Cross-Modal Hashing and Quantization |
|
| 13 | Jurek Leonhardt | 44 | 3 | 54 Efficient Neural Ranking Using Forward Indexes and Lightweight Encoders (2024) Learned Sparse Retrieval Efficiency 54 Efficient Neural Ranking using Forward Indexes (2022) Learned Sparse Retrieval Efficiency |
|
| 15 | Fengling Li 0001 COI: co-author | 42 | 3 | 45 Task-adaptive Asymmetric Deep Cross-modal Hashing (2021) Cross-Modal Hashing and Quantization 45 PIC-CMH: Efficient Prompt-Infused Continual Cross-Modal Hashing (2026) Cross-Modal Hashing and Quantization |
|
| 16 | Meiyu Liang | 42 | 3 | 45 Unsupervised Multimodal Graph Contrastive Semantic Anchor Space Dynamic Knowledge Distillation Network for Cross-Media Hash Retrieval (2024) Cross-Modal Hashing and Quantization 45 Deep Unsupervised Momentum Contrastive Hashing for Cross-modal Retrieval (2023) Cross-Modal Hashing and Quantization |
|
| 17 | Keping Bi | 42 | 3 | 75 A Multi-Granularity-Aware Aspect Learning Model for Multi-Aspect Dense Retrieval (2024) Sparse and Dense Retrieval Encoders 73 Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense Retrieval (2023) Sparse and Dense Retrieval Encoders |
|
| 18 | Jinpeng Wang 0002 | 41 | 3 | 45 Webly Supervised Deep Attentive Quantization (2021) Cross-Modal Hashing and Quantization 45 Hugs Are Better Than Handshakes: Unsupervised Cross-Modal Transformer Hashing with Multi-granularity Alignment (2022) Cross-Modal Hashing and Quantization |
|
| 19 | Yuchen Li 0006 | 41 | 3 | 45 Probe-and-Fetch: Dynamic KV Cache Pruning for Accelerated Long-Context Inference in Web-Scale AI Search (2026) Efficient and Long-Context RAG 45 RankElectra: Semi-supervised Pre-training of Learning-to-Rank Electra for Web-scale Search (2025) Web-Scale Learning to Rank |
|
| 21 | Yanzhao Zhang | 40 | 3 | 53 Unsupervised Semantic Association Learning with Latent Label Inference (2021) Text Embedding and Representation Learning 46 Text Representation Distillation via Information Bottleneck Principle (2023) Text Embedding and Representation Learning |
|
| 22 | Haokun Wen | 40 | 3 | 42 Multi-modal Recommendation with Joint Content and Interaction Augmentation (2025) Search and Recommendation Integration 41 Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 23 | Hao Deng 0011 | 40 | 3 | 42 REG4Rec: Reasoning-Enhanced Generative Model for Large-Scale Recommendation Systems (2026) Item Tokenization and Semantic IDs 42 MMQ: Multimodal Mixture-of-Quantization Tokenization for Semantic ID Generation and User Behavioral Adaptation (2026) Item Tokenization and Semantic IDs |
|
| 24 | Negar Arabzadeh | 40 | 3 | 59 Learning Query-Space Document Representations for High-Recall Retrieval (2023) Dense Bi-Encoder Training and Representation Learning 49 Predicting Efficiency/Effectiveness Trade-offs for Dense vs. Sparse Retrieval Strategy Selection (2021) Efficient Retrieval and Indexing |
|
| 25 | Jieming Zhu | 39 | 3 | 44 CoST: Contrastive Quantization based Semantic Tokenization for Generative Recommendation (2024) Item Tokenization and Semantic IDs 44 Learning Multi-aspect Item Palette: A Semantic Tokenization Framework for Generative Recommendation (2026) Item Tokenization and Semantic IDs |
|
| 26 | Daiting Shi | 39 | 3 | 80 Pre-trained Language Model for Web-scale Retrieval in Baidu Search (2021) Sparse and Dense Retrieval Encoders 55 Pre-trained Language Model-based Retrieval and Ranking for Web Search (2023) Learned Sparse Retrieval Efficiency |
|
| 27 | Qijiong Liu | 39 | 3 | 44 CoST: Contrastive Quantization based Semantic Tokenization for Generative Recommendation (2024) Item Tokenization and Semantic IDs 44 Learning Multi-aspect Item Palette: A Semantic Tokenization Framework for Generative Recommendation (2026) Item Tokenization and Semantic IDs |
|
| 28 | Yupeng Hou | 38 | 3 | 44 Towards Large Generative Recommendation: A Tokenization Perspective (2025) Item Tokenization and Semantic IDs 43 Generating Long Semantic IDs in Parallel for Recommendation (2025) Item Tokenization and Semantic IDs |
|
| 29 | Qiang Luo 0004 | 38 | 3 | 42 MISS: Multi-Modal Tree Indexing and Searching with Lifelong Sequential Behavior for Retrieval Recommendation (2025) Universal Multimodal Embedding and MLLM Retrieval 41 FARM: Frequency-Aware Model for Cross-Domain Live-Streaming Recommendation (2026) Domain Adaptation and Disentangled Representation for CDR |
|
| 30 | Andrew Parry | 38 | 3 | 71 "In-Context Learning" or: How I learned to stop worrying and love "Applied Information Retrieval" (2024) Sparse and Dense Retrieval Encoders 42 Optimal Re-Ranking Depth (2026) Query Performance Prediction |
|
| 31 | Hao Liu 0078 | 38 | 3 | 41 DA-DAN: A Dual Adversarial Domain Adaption Network for Unsupervised Non-overlapping Cross-domain Recommendation (2024) Domain Adaptation and Disentangled Representation for CDR 41 Mitigating Distribution Shifts in Sequential Recommendation: An Invariance Perspective (2025) Contrastive and Self-Supervised Sequential Recommendation |
|
| 32 | Bowen Zheng 0005 | 37 | 3 | 43 Generative Recommender with End-to-End Learnable Item Tokenization (2025) Item Tokenization and Semantic IDs 42 Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation (2024) Generative Recommendation and Item Tokenization |