Adaptive Sparsity Optimization with Learnable Soft Top-K and Per-Term Thresholding for Efficient Retrieval
sigir26-101 · Full research paper · Wentai Xie, Parker Carlson, Shanxiu He, Tao Yang
Learned Sparse Retrieval Efficiency · 50% Efficient Retrieval and Indexing · 25% Learned Sparse Retrieval · 25%
Abstract
Recent work on neural sparse retrieval has demonstrated strong relevance by leveraging Large Language Models (LLMs) for semantic term expansion. However, learned models paired with previous sparsification techniques still yield overly long document and query vectors partly due to a large LLM vocabulary, imposing a serious challenge to retrieval time and space efficiency. This paper proposes a scheme for optimizing model sparsity through a synergy of adaptive strategies, including learnable soft top-??, per-term thresholding, and FLOPs regularization to increase the sparsity of query and document vectors. Experimental results with Lion-SP model on the MS MARCO and BEIR datasets demonstrate that the proposed scheme can outperform the baselines by significantly reducing the average query and document lengths. Our scheme can achieve much shorter retrieval latency and lower storage cost while maintaining highly competitive relevance.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Franco Maria Nardini | 83 | 100 Effective Inference-Free Retrieval for Learned Sparse Representations (2025) Learned Sparse Retrieval Efficiency 100 Efficient Inverted Indexes for Approximate Retrieval over Learned Sparse Representations (2024) Learned Sparse Retrieval Efficiency 100 Pairing Clustered Inverted Indexes with κ-NN Graphs for Fast Approximate Retrieval over Learned Sparse Representations (2024) Learned Sparse Retrieval Efficiency |
|
| 2 | Sebastian Bruch 0001 | 70 | 100 Forward Index Compression for Learned Sparse Retrieval (2026) Learned Sparse Retrieval Efficiency 86 Bridging Dense and Sparse Maximum Inner Product Search (2024) Efficient Retrieval and Indexing 76 Efficient Inverted Indexes for Approximate Retrieval over Learned Sparse Representations (2024) Learned Sparse Retrieval Efficiency |
|
| 3 | Jingtao Zhan | 57 | 100 Joint Optimization of Multi-vector Representation with Product Quantization (2022) Learned Sparse Retrieval Efficiency 72 Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance (2021) Approximate Nearest Neighbor Search 72 Learning Discrete Representations via Constrained Clustering for Effective and Efficient Dense Retrieval (Extended Abstract) (2023) Approximate Nearest Neighbor Search |
Area chair
| Area chair | Match | |
|---|---|---|
| Chenyan Xiong | 41 |
Area-chair candidates (6)
| Chenyan Xiong assigned | 41 | 15 papers | |
| Xiaoyi Zeng | 39 | 11 papers | |
| Xiaoxue Zang | 38 | 4 papers | |
| Jun Xu 0001 | 37 | 2 papers | |
| Fernando Diaz 0001 | 37 | 12 papers | |
| Maolin Wang 0001 | 36 | 15 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Antonio Mallia COI: co-author | 90 | 3 | 100 Faster Learned Sparse Retrieval with Guided Traversal (2022) Learned Sparse Retrieval Efficiency 100 Faster Learned Sparse Retrieval with Block-Max Pruning (2024) Learned Sparse Retrieval Efficiency |
|
| 2 | Franco Maria Nardini assigned | 83 | 3 | 100 Effective Inference-Free Retrieval for Learned Sparse Representations (2025) Learned Sparse Retrieval Efficiency 100 Efficient Inverted Indexes for Approximate Retrieval over Learned Sparse Representations (2024) Learned Sparse Retrieval Efficiency |
|
| 3 | Sebastian Bruch 0001 assigned | 70 | 3 | 100 Forward Index Compression for Learned Sparse Retrieval (2026) Learned Sparse Retrieval Efficiency 86 Bridging Dense and Sparse Maximum Inner Product Search (2024) Efficient Retrieval and Indexing |
|
| 4 | Huiqiang Jiang | 66 | 3 | 76 LeanK: Learnable K Cache Channel Pruning for Efficient Decoding (2025) Learned Sparse Retrieval Efficiency 76 RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval (2025) Learned Sparse Retrieval Efficiency |
|
| 5 | Xueguang Ma | 65 | 3 | 100 SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexes (2023) Learned Sparse Retrieval Efficiency 91 Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations (2021) Learned Sparse Retrieval Efficiency |
|
| 6 | Xiao Wang 0043 | 59 | 3 | 87 Reproducibility, Replicability, and Insights into Dense Multi-Representation Retrieval Models: from ColBERT to Col (2023) Learned Sparse Retrieval Efficiency 82 An Inspection of the Reproducibility and Replicability of TCT-ColBERT (2022) Learned Sparse Retrieval Efficiency |
|
| 7 | Jingtao Zhan assigned | 57 | 3 | 100 Joint Optimization of Multi-vector Representation with Product Quantization (2022) Learned Sparse Retrieval Efficiency 72 Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance (2021) Approximate Nearest Neighbor Search |
|
| 8 | Jurek Leonhardt | 55 | 3 | 90 Efficient Neural Ranking Using Forward Indexes and Lightweight Encoders (2024) Learned Sparse Retrieval Efficiency 90 Efficient Neural Ranking using Forward Indexes (2022) Learned Sparse Retrieval Efficiency |
|
| 9 | Sheng-Chieh Lin | 54 | 3 | 100 SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexes (2023) Learned Sparse Retrieval Efficiency 100 CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval (2023) Learned Sparse Retrieval Efficiency |
|
| 10 | Shengyao Zhuang | 53 | 3 | 95 LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculum (2026) Learned Sparse Retrieval Efficiency 86 Layer-wise Token Compression for Efficient Document Reranking (2026) Learned Sparse Retrieval Efficiency |
|
| 11 | Xinyu Ma 0001 | 47 | 3 | 65 Probe-and-Fetch: Dynamic KV Cache Pruning for Accelerated Long-Context Inference in Web-Scale AI Search (2026) Efficient and Long-Context RAG 58 Replication and Exploration of Generative Retrieval over Dynamic Corpora (2025) Generative Retrieval |
|
| 12 | Corby Rosset | 44 | 3 | 86 Optimizing Query Evaluations Using Reinforcement Learning for Web Search (2018) Efficient Retrieval and Indexing 63 Dodo: Dynamic Contextual Compression for Decoder-only LMs (2024) Learned Sparse Retrieval Efficiency |
|
| 13 | Negar Arabzadeh | 44 | 3 | 76 Predicting Efficiency/Effectiveness Trade-offs for Dense vs. Sparse Retrieval Strategy Selection (2021) Efficient Retrieval and Indexing 48 Learning Query-Space Document Representations for High-Recall Retrieval (2023) Dense Bi-Encoder Training and Representation Learning |
|
| 14 | Daiting Shi | 43 | 3 | 75 Pre-trained Language Model-based Retrieval and Ranking for Web Search (2023) Learned Sparse Retrieval Efficiency 72 Pre-trained Language Model based Ranking in Baidu Search (2021) Learned Sparse Retrieval Efficiency |
|
| 15 | Yuchen Li 0006 | 42 | 3 | 65 Probe-and-Fetch: Dynamic KV Cache Pruning for Accelerated Long-Context Inference in Web-Scale AI Search (2026) Efficient and Long-Context RAG 43 COLTR: Semi-Supervised Learning to Rank With Co-Training and Over-Parameterization for Web Search (2023) Web-Scale Learning to Rank |
|
| 17 | Keping Bi | 41 | 3 | 55 Reproducibility Analysis and Enhancements for Multi-aspect Dense Retriever with Aspect Learning (2024) Retrieval Toolkits and Reproducibility 52 A Multi-Granularity-Aware Aspect Learning Model for Multi-Aspect Dense Retrieval (2024) Sparse and Dense Retrieval Encoders |
|
| 18 | Ruqing Zhang 0001 | 41 | 3 | 50 PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders 50 B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders |
|
| 19 | Venktesh V | 39 | 3 | 86 Breaking the Lens of the Telescope: Online Relevance Estimation over Large Retrieval Sets (2025) Learned Sparse Retrieval Efficiency 43 TagRec++: Hierarchical Label Aware Attention Network for Question Categorization (2024) Text Embedding and Representation Learning |
|
| 21 | Bo Chen 0023 | 39 | 3 | 71 Efficiency Unleashed: Inference Acceleration for LLM-based Recommender Systems with Speculative Decoding (2025) Efficiency and Deployment 39 Towards efficient and effective unlearning of large language models for recommendation (2025) Efficiency and Deployment |
|
| 22 | Amir Ingber | 39 | 3 | 76 SDR: Efficient Neural Re-ranking using Succinct Document Representation (2022) Learned Sparse Retrieval Efficiency 69 Bridging Dense and Sparse Maximum Inner Product Search (2024) Approximate Nearest Neighbor Search |
|
| 23 | Peitian Zhang | 39 | 3 | 81 Hybrid Inverted Index Is a Robust Accelerator for Dense Retrieval (2023) Efficient Retrieval and Indexing 37 INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning (2024) LLM-Based Reranking |
|
| 24 | Jialong Wu 0007 | 38 | 3 | 76 SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation (2025) Learned Sparse Retrieval Efficiency 41 AdaCQR: Enhancing Query Reformulation for Conversational Search via Sparse and Dense Retrieval Alignment (2025) Query Reformulation and Expansion |
|
| 25 | Yupeng Hou | 38 | 3 | 43 Large Language Models are Zero-Shot Rankers for Recommender Systems (2024) Efficiency and Deployment 38 RecBole 2.0: Towards a More Up-to-Date Recommendation Library (2022) Recommendation Systems and Evaluation |
|
| 27 | Shuaiqiang Wang | 38 | 3 | 43 COLTR: Semi-Supervised Learning to Rank With Co-Training and Over-Parameterization for Web Search (2023) Web-Scale Learning to Rank 43 GS2P: A Generative Pre-trained Learning to Rank Model with Over-parameterization for Web-Scale Search (Extended Abstract) (2024) Web-Scale Learning to Rank |
|
| 28 | Han Li 0005 | 38 | 3 | 41 LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application (2025) Efficiency and Deployment 41 Comprehensive List Generation for Multi-Generator Reranking (2025) Recommendation with Search and Ranking |
|
| 29 | Jizhi Zhang | 37 | 3 | 43 Navigating Large Language Models for Recommendation: From Architecture to Learning Paradigms and Deployment (2025) Efficiency and Deployment 39 Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation (2023) Preference Alignment and Optimization |
|
| 30 | Andrew Parry | 37 | 3 | 45 "In-Context Learning" or: How I learned to stop worrying and love "Applied Information Retrieval" (2024) Sparse and Dense Retrieval Encoders 42 Optimal Re-Ranking Depth (2026) Query Performance Prediction |
|
| 31 | Bowen Zheng 0005 | 37 | 3 | 38 Towards a More User-Friendly and Easy-to-Use Benchmark Library for Recommender Systems (2023) Recommendation Systems and Evaluation 37 Generative Recommender with End-to-End Learnable Item Tokenization (2025) Item Tokenization and Semantic IDs |
|
| 33 | Yang Zhang 0072 | 37 | 3 | 43 Preliminary Study on Incremental Learning for Large Language Model-based Recommender Systems (2024) Efficiency and Deployment 39 Exact and Efficient Unlearning for Large Language Model-Based Recommendation (2025) Efficiency and Deployment |
|
| 35 | Yimeng Bai | 37 | 3 | 38 LabelCraft: Empowering Short Video Recommendations with Automated Label Crafting (2024) Recommendation Systems and Evaluation 38 Leveraging Watch-time Feedback for Short-Video Recommendations: A Causal Labeling Framework (2023) Recommendation Systems and Evaluation |