Constructing Hard-Positive Query-Document Pairs for Dense Retrieval via Phrase Representativeness
Abstract
Dense retrieval usually fails in two ways: ranking non-relevant documents too high, or ranking relevant documents too low. We focus on the second case from the query side: documents that are genuinely relevant but receive low retrieval scores under certain query formulations, forming hard-positive query–document pairs. To study this failure mode systematically, we build on recent token-alignment work, which analyzes vocabulary logits obtained by projecting retriever representations through an architecture-specific token prediction head and shows that low overlap among top-ranked logit tokens can cause relevant documents to be scored low. This suggests a practical route to hard positives: generate relevant queries that rely on document phrases that receive low ranks under these logits. We therefore define model-specific Token and Phrase Representativeness Scores (TRS/PRS) to discover tokens and key phrases that appear in a document but are poorly expressed by its embedding. Using high-PRS phrases as anchors, we automatically construct challenging yet relevant queries, yielding hard-positive query–document pairs. Experiments across multiple retrievers and datasets show that PRS-anchored queries induce substantially larger drops for dense retrievers than other query construction strategies, revealing a dense-specific weakness. Moreover, when mixed with standard hard negatives during contrastive fine-tuning, these hard positives improve retrieval on our constructed hard-positive benchmark and can also improve standard in-domain retrieval benchmarks. Our code is publicly available at https://github.com/wzy2001wzy/HardPositive.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Ruqing Zhang 0001 | 48 | 78 PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders 78 B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders 75 Pre-train a Discriminative Text Encoder for Dense Retrieval via Contrastive Span Prediction (2022) Dense Bi-Encoder Training and Representation Learning |
|
| 2 | Zheng Zhang 0006 | 46 | 46 Enhancing Cross-Modal Retrieval via Visual-Textual Prompt Hashing (2024) Cross-Modal Hashing and Quantization 46 Semantics-Reconstructing Hashing for Cross-Modal Retrieval (2020) Cross-Modal Hashing and Quantization 46 Cross-Domain Transfer Hashing for Efficient Cross-Modal Retrieval (2024) Cross-Modal Hashing and Quantization |
Area chair
| Area chair | Match | |
|---|---|---|
| Chenyan Xiong | 42 |
Area-chair candidates (2)
| Chenyan Xiong assigned | 42 | 15 papers | |
| Xiaoyi Zeng | 38 | 11 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Xiao Wang 0043 | 50 | 3 | 71 Unsupervised Dense Retrieval with Conterfactual Contrastive Learning (2026) Robustness and Adversarial IR 54 An Inspection of the Reproducibility and Replicability of TCT-ColBERT (2022) Learned Sparse Retrieval Efficiency |
|
| 2 | Xueguang Ma | 50 | 3 | 70 To Interpolate or not to Interpolate: PRF, Dense and Sparse Retrievers (2022) Query Reformulation and Expansion 65 Fine-Tuning LLaMA for Multi-Stage Text Retrieval (2024) Sparse and Dense Retrieval Encoders |
|
| 3 | Xinyu Ma 0001 | 50 | 3 | 78 PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders 78 Pre-train a Discriminative Text Encoder for Dense Retrieval via Contrastive Span Prediction (2022) Sparse and Dense Retrieval Encoders |
|
| 4 | Ruqing Zhang 0001 assigned | 48 | 3 | 78 PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders 78 B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders |
|
| 5 | Shengyao Zhuang | 48 | 3 | 65 PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval (2024) Text Embedding and Representation Learning 63 Dealing with Typos for BERT-based Passage Retrieval and Ranking (2021) Robustness and Adversarial IR |
|
| 6 | Sheng-Chieh Lin | 47 | 3 | 75 Aggretriever: A Simple Approach to Aggregate Textual Representations for Robust Dense Passage Retrieval (2023) Dense Bi-Encoder Training and Representation Learning 75 A Dense Representation Framework for Lexical and Semantic Matching (2023) Dense Bi-Encoder Training and Representation Learning |
|
| 7 | Jurek Leonhardt | 47 | 3 | 57 Efficient Neural Ranking using Forward Indexes (2022) Learned Sparse Retrieval Efficiency 57 Efficient Neural Ranking Using Forward Indexes and Lightweight Encoders (2024) Learned Sparse Retrieval Efficiency |
|
| 8 | Jingtao Zhan | 46 | 3 | 100 Optimizing Dense Retrieval Model Training with Hard Negatives (2021) Dense Bi-Encoder Training and Representation Learning 78 Scaling Laws For Dense Retrieval (2024) Sparse and Dense Retrieval Encoders |
|
| 9 | Zheng Zhang 0006 assigned | 46 | 3 | 46 Enhancing Cross-Modal Retrieval via Visual-Textual Prompt Hashing (2024) Cross-Modal Hashing and Quantization 46 Semantics-Reconstructing Hashing for Cross-Modal Retrieval (2020) Cross-Modal Hashing and Quantization |
|
| 10 | Antonio Mallia | 45 | 3 | 54 Learning Passage Impacts for Inverted Indexes (2021) Learned Sparse Retrieval Efficiency 54 Improved Learned Sparse Retrieval with Corpus-Specific Vocabularies (2024) Learned Sparse Retrieval Efficiency |
|
| 11 | Franco Maria Nardini | 45 | 3 | 54 Expansion via Prediction of Importance with Contextualization (2020) Sparse and Dense Retrieval Encoders 48 ReNeuIR at SIGIR 2026: The Fifth Workshop on Reaching Efficiency in Neural Information Retrieval (2026) Learned Sparse Retrieval Efficiency |
|
| 12 | Sebastian Bruch 0001 | 45 | 3 | 100 An Analysis of Fusion Functions for Hybrid Retrieval (2024) Dense Bi-Encoder Training and Representation Learning 48 Efficient Inverted Indexes for Approximate Retrieval over Learned Sparse Representations (2024) Learned Sparse Retrieval Efficiency |
|
| 13 | Jing Liu 0022 | 43 | 3 | 100 Dense Text Retrieval Based on Pretrained Language Models: A Survey (2024) Dense Bi-Encoder Training and Representation Learning 100 RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2021) Dense Bi-Encoder Training and Representation Learning |
|
| 14 | Huiqiang Jiang | 42 | 3 | 48 RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval (2025) Learned Sparse Retrieval Efficiency 48 LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models (2023) Learned Sparse Retrieval Efficiency |
|
| 15 | Yuchen Li 0006 | 42 | 3 | 47 S2phere: Semi-Supervised Pre-training for Web Search over Heterogeneous Learning to Rank Data (2023) Web-Scale Learning to Rank 47 RankElectra: Semi-supervised Pre-training of Learning-to-Rank Electra for Web-scale Search (2025) Web-Scale Learning to Rank |
|
| 17 | Meiyu Liang | 42 | 3 | 46 Unsupervised Multimodal Graph Contrastive Semantic Anchor Space Dynamic Knowledge Distillation Network for Cross-Media Hash Retrieval (2024) Cross-Modal Hashing and Quantization 46 Deep Unsupervised Momentum Contrastive Hashing for Cross-modal Retrieval (2023) Cross-Modal Hashing and Quantization |
|
| 18 | Keping Bi | 41 | 3 | 100 CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrieval (2025) Dense Bi-Encoder Training and Representation Learning 58 L2R: Lifelong Learning for First-stage Retrieval with Backward-Compatible Representations (2023) Efficient Retrieval and Indexing Systems › Others |
|
| 19 | Fengling Li 0001 | 41 | 3 | 46 Noise-Robust Generative Hashing for Cross-Modal Retrieval (2026) Cross-Modal Hashing and Quantization 46 Prompt-Driven Bit Extension Hashing for Continual Cross-Modal Retrieval (2026) Cross-Modal Hashing and Quantization |
|
| 20 | Negar Arabzadeh | 41 | 3 | 100 Learning Query-Space Document Representations for High-Recall Retrieval (2023) Dense Bi-Encoder Training and Representation Learning 54 Predicting Efficiency/Effectiveness Trade-offs for Dense vs. Sparse Retrieval Strategy Selection (2021) Efficient Retrieval and Indexing |
|
| 21 | Jinpeng Wang 0002 | 38 | 3 | 46 Webly Supervised Deep Attentive Quantization (2021) Cross-Modal Hashing and Quantization 46 Hugs Bring Double Benefits: Unsupervised Cross-Modal Hashing with Multi-granularity Aligned Transformers (2024) Cross-Modal Hashing and Quantization |
|
| 22 | Daiting Shi | 38 | 3 | 78 Pre-trained Language Model for Web-scale Retrieval in Baidu Search (2021) Sparse and Dense Retrieval Encoders 61 Pre-trained Language Model-based Retrieval and Ranking for Web Search (2023) Learned Sparse Retrieval Efficiency |
|
| 23 | Yanzhao Zhang COI: co-author | 38 | 3 | 71 Unsupervised Semantic Association Learning with Latent Label Inference (2021) Text Embedding and Representation Learning 45 ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Reranking (2026) LLM-Based Reranking |
|
| 25 | Haokun Wen | 38 | 3 | 40 FiRE: Enhancing MLLMs with Fine-Grained Context Learning for Complex Image Retrieval (2025) Universal Multimodal Embedding and MLLM Retrieval 40 Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval (2024) Universal Multimodal Embedding and MLLM Retrieval |
|
| 26 | Jieming Zhu | 38 | 3 | 43 CoST: Contrastive Quantization based Semantic Tokenization for Generative Recommendation (2024) Item Tokenization and Semantic IDs 43 Learning Multi-aspect Item Palette: A Semantic Tokenization Framework for Generative Recommendation (2026) Item Tokenization and Semantic IDs |
|
| 27 | Hao Deng 0011 | 38 | 3 | 41 MMQ: Multimodal Mixture-of-Quantization Tokenization for Semantic ID Generation and User Behavioral Adaptation (2026) Item Tokenization and Semantic IDs 40 REG4Rec: Reasoning-Enhanced Generative Model for Large-Scale Recommendation Systems (2026) Item Tokenization and Semantic IDs |
|
| 28 | Venktesh V | 38 | 3 | 58 TagRec++: Hierarchical Label Aware Attention Network for Question Categorization (2024) Text Embedding and Representation Learning 58 TagRec: Automated Tagging of Questions with Hierarchical Learning Taxonomy (2021) Text Embedding and Representation Learning |
|
| 29 | Chen Zhao 0013 | 37 | 3 | 71 Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective (2025) Text Embedding and Representation Learning 66 Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation (2021) Dense Bi-Encoder Training and Representation Learning |
|
| 30 | Shuaiqiang Wang | 37 | 3 | 47 GS2P: A Generative Pre-trained Learning to Rank Model with Over-parameterization for Web-Scale Search (2023) Web-Scale Learning to Rank 47 GS2P: a generative pre-trained learning to rank model with over-parameterization for web-scale search (2024) Web-Scale Learning to Rank |
|
| 31 | Andrew Parry | 37 | 3 | 52 "In-Context Learning" or: How I learned to stop worrying and love "Applied Information Retrieval" (2024) Sparse and Dense Retrieval Encoders 42 MechIR: A Mechanistic Interpretability Framework for Information Retrieval (2025) Robustness and Adversarial IR |
|
| 32 | Yong He 0009 | 37 | 3 | 41 Uncertainty-based Heterogeneous Privileged Knowledge Distillation for Recommendation System (2023) Domain Adaptation and Disentangled Representation for CDR 41 The Devil is in the Sources! Knowledge Enhanced Cross-Domain Recommendation in an Information Bottleneck Perspective (2024) Domain Adaptation and Disentangled Representation for CDR |