GIGP+: A CPU-GPU Co-Processing Engine for Multi-Vector Retrieval
sigir26-218 · Full research paper · Zheng Bian, Man Lung Yiu, Bo Tang
Learned Sparse Retrieval Efficiency · 100%
Abstract
Multi-vector retrieval models (e.g., ColBERTv2) offer high retrieval accuracy but suffer from efficiency problems at scale. Recently, several methods have been developed to enhance the efficiency of multi-vector retrieval. On one hand, the state-of-the-art GPU-based method PLAID-GPU exploits the massive parallelism of the GPU to accelerate computation, but it needs to process a considerable amount (e.g., ten thousand) of document candidates. On the other hand, the state-of-the-art (SOTA) CPU-based method IGP employs a more effective strategy to reduce the number of candidates, but fails to utilize the massive parallelism of the GPU. To get the best of both worlds, we propose GIGP+, a GPU-based method designed to achieve high parallelism and low computational overhead. Our contributions are: (1) an efficient candidate generation kernel that enjoys parallelism while retaining the effectiveness of IGP, (2) a score reordering mechanism that reduces the synchronization overhead and (3) a scheduling strategy for efficient batch processing. Our experiments demonstrate that GIGP+ achieves a 11.0× improvement in query per second (QPS) and reduces latency by 7.6× compared to PLAID-GPU, while maintaining equivalent retrieval accuracy. As for cloud pricing, GIGP+ delivers a 2.3× improvement in queries per dollar over SOTA CPU-based solutions.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Antonio Mallia | 69 | 81 Learning Passage Impacts for Inverted Indexes (2021) Learned Sparse Retrieval Efficiency 81 Improved Learned Sparse Retrieval with Corpus-Specific Vocabularies (2024) Learned Sparse Retrieval Efficiency 76 Scalable K-Means Guided Partitioning for Block-based Sparse Document Retrieval (2026) Learned Sparse Retrieval Efficiency |
|
| 2 | Huiqiang Jiang | 68 | 100 RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval (2025) Learned Sparse Retrieval Efficiency 100 SCBench: A KV Cache-Centric Analysis of Long-Context Methods (2025) Learned Sparse Retrieval Efficiency 100 RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference (2026) Learned Sparse Retrieval Efficiency |
Area chair
| Area chair | Match | |
|---|---|---|
| Xiaoyi Zeng | 36 |
Area-chair candidates (4)
| Chenyan Xiong | 36 | 15 papers | |
| Xiaoyi Zeng assigned | 36 | 11 papers | |
| Maolin Wang 0001 | 35 | 15 papers | |
| Xiaoxue Zang | 35 | 4 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Franco Maria Nardini | 75 | 3 | 100 ReNeuIR at SIGIR 2026: The Fifth Workshop on Reaching Efficiency in Neural Information Retrieval (2026) Learned Sparse Retrieval Efficiency 100 ReNeuIR at SIGIR 2023: The Second Workshop on Reaching Efficiency in Neural Information Retrieval (2023) Learned Sparse Retrieval Efficiency |
|
| 2 | Antonio Mallia assigned | 69 | 3 | 81 Learning Passage Impacts for Inverted Indexes (2021) Learned Sparse Retrieval Efficiency 81 Improved Learned Sparse Retrieval with Corpus-Specific Vocabularies (2024) Learned Sparse Retrieval Efficiency |
|
| 3 | Huiqiang Jiang assigned | 68 | 3 | 100 RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval (2025) Learned Sparse Retrieval Efficiency 100 SCBench: A KV Cache-Centric Analysis of Long-Context Methods (2025) Learned Sparse Retrieval Efficiency |
|
| 4 | Sebastian Bruch 0001 | 60 | 3 | 100 Efficient Inverted Indexes for Approximate Retrieval over Learned Sparse Representations (2024) Learned Sparse Retrieval Efficiency 100 ReNeuIR at SIGIR 2023: The Second Workshop on Reaching Efficiency in Neural Information Retrieval (2023) Learned Sparse Retrieval Efficiency |
|
| 5 | Xueguang Ma | 54 | 3 | 81 LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculum (2026) Learned Sparse Retrieval Efficiency 76 SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexes (2023) Learned Sparse Retrieval Efficiency |
|
| 6 | Xiao Wang 0043 | 50 | 3 | 74 Reproducibility, Replicability, and Insights into Dense Multi-Representation Retrieval Models: from ColBERT to Col (2023) Learned Sparse Retrieval Efficiency 73 An Inspection of the Reproducibility and Replicability of TCT-ColBERT (2022) Learned Sparse Retrieval Efficiency |
|
| 7 | Sheng-Chieh Lin | 47 | 3 | 76 SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexes (2023) Learned Sparse Retrieval Efficiency 76 CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval (2023) Learned Sparse Retrieval Efficiency |
|
| 8 | Jurek Leonhardt | 47 | 3 | 72 Efficient Neural Ranking Using Forward Indexes and Lightweight Encoders (2024) Learned Sparse Retrieval Efficiency 72 Efficient Neural Ranking using Forward Indexes (2022) Learned Sparse Retrieval Efficiency |
|
| 9 | Shengyao Zhuang | 44 | 3 | 74 LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculum (2026) Learned Sparse Retrieval Efficiency 69 Layer-wise Token Compression for Efficient Document Reranking (2026) Learned Sparse Retrieval Efficiency |
|
| 10 | Jingtao Zhan | 43 | 3 | 76 Joint Optimization of Multi-vector Representation with Product Quantization (2022) Learned Sparse Retrieval Efficiency 51 Scaling Laws For Dense Retrieval (2024) Sparse and Dense Retrieval Encoders |
|
| 11 | Xinyu Ma 0001 | 42 | 3 | 66 Probe-and-Fetch: Dynamic KV Cache Pruning for Accelerated Long-Context Inference in Web-Scale AI Search (2026) Efficient and Long-Context RAG 51 PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders |
|
| 12 | Yuchen Li 0006 | 40 | 3 | 66 Probe-and-Fetch: Dynamic KV Cache Pruning for Accelerated Long-Context Inference in Web-Scale AI Search (2026) Efficient and Long-Context RAG 43 GS2P: a generative pre-trained learning to rank model with over-parameterization for web-scale search (2024) Web-Scale Learning to Rank |
|
| 13 | Negar Arabzadeh | 39 | 3 | 58 Predicting Efficiency/Effectiveness Trade-offs for Dense vs. Sparse Retrieval Strategy Selection (2021) Efficient Retrieval and Indexing 48 Learning Query-Space Document Representations for High-Recall Retrieval (2023) Dense Bi-Encoder Training and Representation Learning |
|
| 14 | Daiting Shi | 38 | 3 | 73 Pre-trained Language Model-based Retrieval and Ranking for Web Search (2023) Learned Sparse Retrieval Efficiency 72 Pre-trained Language Model based Ranking in Baidu Search (2021) Learned Sparse Retrieval Efficiency |
|
| 15 | Bo Chen 0023 | 38 | 3 | 72 Efficiency Unleashed: Inference Acceleration for LLM-based Recommender Systems with Speculative Decoding (2025) Efficiency and Deployment 39 Large language models make sample-efficient recommender systems (2025) Efficiency and Deployment |
|
| 16 | Corby Rosset | 38 | 3 | 70 Optimizing Query Evaluations Using Reinforcement Learning for Web Search (2018) Efficient Retrieval and Indexing 65 Dodo: Dynamic Contextual Compression for Decoder-only LMs (2024) Learned Sparse Retrieval Efficiency |
|
| 17 | Zheng Zhang 0006 | 37 | 3 | 37 Deep collaborative graph hashing for discriminative image retrieval (2023) Cross-Modal Hashing and Quantization 37 Flexible Discrete Multi-view Hashing with Collective Latent Feature Learning (2020) Cross-Modal Hashing and Quantization |
|
| 18 | Ruqing Zhang 0001 | 37 | 3 | 51 PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders 51 B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders |
|
| 19 | Jialong Wu 0007 | 37 | 3 | 100 SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation (2025) Learned Sparse Retrieval Efficiency 38 AdaCQR: Enhancing Query Reformulation for Conversational Search via Sparse and Dense Retrieval Alignment (2025) Query Reformulation and Expansion |
|
| 20 | Jieming Zhu | 37 | 3 | 40 FANS: Fast Non-Autoregressive Sequence Generation for Item List Continuation (2023) Efficient and Long-Sequence Sequential Recommendation 40 Can LLMs Outshine Conventional Recommenders? A Comparative Evaluation (2025) Efficiency and Deployment |
|
| 21 | Hao Deng 0011 | 36 | 3 | 39 REG4Rec: Reasoning-Enhanced Generative Model for Large-Scale Recommendation Systems (2026) Item Tokenization and Semantic IDs 38 CSMF: Cascaded Selective Mask Fine-Tuning for Multi-Objective Embedding-Based Retrieval (2025) Multi-Task and Multi-Objective Optimization |
|
| 22 | Amir Ingber | 36 | 3 | 100 SDR: Efficient Neural Re-ranking using Succinct Document Representation (2022) Learned Sparse Retrieval Efficiency 72 Bridging Dense and Sparse Maximum Inner Product Search (2024) Approximate Nearest Neighbor Search |
|
| 25 | Qijiong Liu | 36 | 3 | 40 FANS: Fast Non-Autoregressive Sequence Generation for Item List Continuation (2023) Efficient and Long-Sequence Sequential Recommendation 39 Learning Multi-aspect Item Palette: A Semantic Tokenization Framework for Generative Recommendation (2026) Item Tokenization and Semantic IDs |
|
| 26 | Yupeng Hou | 36 | 3 | 41 Large Language Models are Zero-Shot Rankers for Recommender Systems (2024) Efficiency and Deployment 39 Towards Large Generative Recommendation: A Tokenization Perspective (2025) Item Tokenization and Semantic IDs |
|
| 27 | Bowen Zheng 0005 | 36 | 3 | 39 Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation (2024) Generative Recommendation and Item Tokenization 39 Generative Recommender with End-to-End Learnable Item Tokenization (2025) Item Tokenization and Semantic IDs |
|
| 28 | Han Li 0005 | 35 | 3 | 42 LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application (2025) Efficiency and Deployment 40 DualGR: Generative Retrieval with Long and Short-Term Interests Modeling (2026) Generative Retrieval and Ranking |
|
| 29 | Keping Bi | 35 | 3 | 48 CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrieval (2025) Dense Bi-Encoder Training and Representation Learning 47 A Multi-Granularity-Aware Aspect Learning Model for Multi-Aspect Dense Retrieval (2024) Sparse and Dense Retrieval Encoders |
|
| 30 | Jizhi Zhang | 35 | 3 | 44 Navigating Large Language Models for Recommendation: From Architecture to Learning Paradigms and Deployment (2025) Efficiency and Deployment 40 CoLLM: Integrating Collaborative Embeddings Into Large Language Models for Recommendation (2025) Collaborative-Semantic Representation Alignment |
|
| 31 | Fengling Li 0001 | 35 | 3 | 37 Unified stable and generalizable online hashing for cross-modal retrieval (2026) Cross-Modal Hashing and Quantization 37 One for more: Structured Multi-Modal Hashing for multiple multimedia retrieval tasks (2023) Cross-Modal Hashing and Quantization |
|
| 33 | Hao Liu 0078 | 35 | 3 | 38 Mitigating Distribution Shifts in Sequential Recommendation: An Invariance Perspective (2025) Contrastive and Self-Supervised Sequential Recommendation 36 WeaveRec: An LLM-Based Cross-Domain Sequential Recommendation Framework with Model Merging (2026) Efficiency and Deployment |