Learning to Retrieve from Agent Trajectories
Abstract
Information retrieval (IR) systems have traditionally been designed and trained for human users, with learning-to-rank methods relying heavily on large-scale human interaction logs such as clicks and dwell time. With the rapid emergence of large language model (LLM) powered search agents, however, retrieval is increasingly consumed by agents rather than human beings, and is embedded as a core component within multi-turn reasoning and action loops. In this setting, retrieval models trained under human-centric assumptions can be mismatched with the way agents issue intermediate queries and consume results. In this work, we argue that retrieval models for agentic search should be trained directly from agent interaction data. We study learning to retrieve from agent trajectories as a trajectory-supervised training setting, where supervision is derived from multi-step agent interactions. Through a systematic analysis of search agent trajectories, we identify key behavioral signals that reveal document utility, including browsing actions, unbrowsed rejections, and post-browse reasoning traces. Guided by these insights, we propose LRAT, a simple yet effective framework that mines high-quality retrieval supervision from agent trajectories and incorporates relevance intensity through weighted optimization. To instantiate this setting at scale, we deploy the Tongyi-DeepResearch-30B model on 10K InfoSeekQA queries with four retrievers, collecting 26,482 agent trajectories and constructing 91,713 training pairs. Extensive experiments on both in-domain and out-of-domain deep research benchmarks demonstrate that retrievers trained with LRAT consistently improve evidence recall, end-to-end task success, and execution efficiency across diverse agent architectures and scales. Our results highlight agent trajectories as a practical and scalable supervision source for retrieval in agentic search.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Xinyu Ma 0001 | 56 | 86 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Reasoning-Intensive Retrieval and Ranking 60 ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (2026) LLM-Based Reranking 53 PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders |
|
| 2 | Jurek Leonhardt | 55 | 58 Data Augmentation for Sample Efficient and Robust Document Ranking (2024) Ranking Distillation and Adaptation 54 Efficient Neural Ranking using Forward Indexes (2022) Learned Sparse Retrieval Efficiency 54 Efficient Neural Ranking Using Forward Indexes and Lightweight Encoders (2024) Learned Sparse Retrieval Efficiency |
|
| 3 | Yuchen Li 0006 | 49 | 60 S2phere: Semi-Supervised Pre-training for Web Search over Heterogeneous Learning to Rank Data (2023) Web-Scale Learning to Rank 53 Retain to Refine: Adaptive Online Question Answering via Query Routing and Long-Short Memory (2026) Agentic and Deep Search RAG 49 GS2P: A Generative Pre-trained Learning to Rank Model with Over-parameterization for Web-Scale Search (Extended Abstract) (2024) Web-Scale Learning to Rank |
Area chair
| Area chair | Match | |
|---|---|---|
| Fernando Diaz 0001 | 43 |
Area-chair candidates (7)
| Chenyan Xiong COI: co-author | 47 | 15 papers | |
| Kam-Fai Wong COI: co-author | 44 | 6 papers | |
| Xiaoxue Zang COI: co-author | 44 | 4 papers | |
| Fernando Diaz 0001 assigned | 43 | 12 papers | |
| Hongjin Qian COI: co-author | 43 | 12 papers | |
| Jun Xu 0001 COI: author | 43 | 2 papers | |
| Zhijing Wu 0001 | 43 | 9 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Xinyu Ma 0001 assigned | 56 | 3 | 86 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Reasoning-Intensive Retrieval and Ranking 60 ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (2026) LLM-Based Reranking |
|
| 2 | Jurek Leonhardt assigned | 55 | 3 | 58 Data Augmentation for Sample Efficient and Robust Document Ranking (2024) Ranking Distillation and Adaptation 54 Efficient Neural Ranking using Forward Indexes (2022) Learned Sparse Retrieval Efficiency |
|
| 3 | Shengyao Zhuang | 55 | 3 | 75 R2LLMs: Retrieval and Ranking with LLMs (2025) Reasoning-Intensive Retrieval and Ranking 72 Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning (2026) Reasoning-Intensive Retrieval and Ranking |
|
| 4 | Jing Liu 0022 COI: co-author | 52 | 3 | 70 RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking (2021) Dense Bi-Encoder Training and Representation Learning 55 Dense Text Retrieval Based on Pretrained Language Models: A Survey (2024) Dense Bi-Encoder Training and Representation Learning |
|
| 5 | Xueguang Ma | 51 | 3 | 79 R2LLMs: Retrieval and Ranking with LLMs (2025) Reasoning-Intensive Retrieval and Ranking 61 To Interpolate or not to Interpolate: PRF, Dense and Sparse Retrievers (2022) Query Reformulation and Expansion |
|
| 6 | Xiao Wang 0043 | 50 | 3 | 52 Unsupervised Dense Retrieval with Conterfactual Contrastive Learning (2026) Robustness and Adversarial IR 49 Neural Pseudo-Relevance Feedback Models for Sparse and Dense Retrieval (2022) Query Reformulation and Expansion |
|
| 7 | Ruqing Zhang 0001 COI: co-author | 50 | 3 | 53 PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders 53 B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders |
|
| 8 | Keping Bi | 50 | 3 | 55 CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrieval (2025) Dense Bi-Encoder Training and Representation Learning 52 L2R: Lifelong Learning for First-stage Retrieval with Backward-Compatible Representations (2023) Efficient Retrieval and Indexing Systems › Others |
|
| 9 | Yuchen Li 0006 assigned | 49 | 3 | 60 S2phere: Semi-Supervised Pre-training for Web Search over Heterogeneous Learning to Rank Data (2023) Web-Scale Learning to Rank 53 Retain to Refine: Adaptive Online Question Answering via Query Routing and Long-Short Memory (2026) Agentic and Deep Search RAG |
|
| 10 | Chen Zhao 0013 | 49 | 3 | 69 Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems (2026) Reasoning-Intensive Retrieval and Ranking 60 Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval (2021) Multi-hop and Complex Reasoning QA |
|
| 11 | Sheng-Chieh Lin | 48 | 3 | 52 Aggretriever: A Simple Approach to Aggregate Textual Representations for Robust Dense Passage Retrieval (2023) Dense Bi-Encoder Training and Representation Learning 52 A Dense Representation Framework for Lexical and Semantic Matching (2023) Dense Bi-Encoder Training and Representation Learning |
|
| 12 | Jingtao Zhan | 48 | 3 | 55 Optimizing Dense Retrieval Model Training with Hard Negatives (2021) Dense Bi-Encoder Training and Representation Learning 53 Scaling Laws For Dense Retrieval (2024) Sparse and Dense Retrieval Encoders |
|
| 13 | Daiting Shi COI: co-author | 48 | 3 | 58 Pre-trained Language Model-based Retrieval and Ranking for Web Search (2023) Learned Sparse Retrieval Efficiency 53 Pre-trained Language Model for Web-scale Retrieval in Baidu Search (2021) Sparse and Dense Retrieval Encoders |
|
| 14 | Weiwei Sun 0001 | 47 | 3 | 86 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Reasoning-Intensive Retrieval and Ranking 75 TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy (2025) Reasoning-Intensive Retrieval and Ranking |
|
| 16 | Negar Arabzadeh | 47 | 3 | 55 Learning Query-Space Document Representations for High-Recall Retrieval (2023) Dense Bi-Encoder Training and Representation Learning 52 Predicting Efficiency/Effectiveness Trade-offs for Dense vs. Sparse Retrieval Strategy Selection (2021) Efficient Retrieval and Indexing |
|
| 17 | Sebastian Bruch 0001 | 47 | 3 | 55 An Analysis of Fusion Functions for Hybrid Retrieval (2024) Dense Bi-Encoder Training and Representation Learning 49 Training Efficient Tree-Based Models for Document Ranking (2013) Web-Scale Learning to Rank |
|
| 18 | Venktesh V | 45 | 3 | 53 The Surprising Effectiveness of Rankers trained on Expanded Queries (2024) Evidence Selection and Context Refinement 50 TagRec++: Hierarchical Label Aware Attention Network for Question Categorization (2024) Text Embedding and Representation Learning |
|
| 19 | Shuaiqiang Wang COI: co-author | 45 | 3 | 54 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Text Embedding and Representation Learning 49 GS2P: A Generative Pre-trained Learning to Rank Model with Over-parameterization for Web-Scale Search (Extended Abstract) (2024) Web-Scale Learning to Rank |
|
| 20 | Adam Jatowt | 45 | 3 | 75 BracketRank: Large Language Model Document Ranking via Reasoning-based Competitive Elimination (2026) Reasoning-Intensive Retrieval and Ranking 58 TempRetriever: Fusion-based Temporal Dense Passage Retrieval for Time-Sensitive Questions (2026) Domain-Specific and Structured Retrieval |
|
| 21 | Zhen Qin 0001 | 45 | 3 | 58 Learning List-Level Domain-Invariant Representations for Ranking (2023) Ranking Distillation and Adaptation 58 Multi-Task Learning for Email Search Ranking with Auxiliary Query Clustering (2018) Domain-Specific and Structured Retrieval |
|
| 22 | Corby Rosset | 45 | 3 | 51 Generic Intent Representation in Web Search (2019) Text Embedding and Representation Learning 44 Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories (2023) Retrieval-Augmented Generation › Others |
|
| 23 | Andrew Parry | 44 | 3 | 58 "In-Context Learning" or: How I learned to stop worrying and love "Applied Information Retrieval" (2024) Sparse and Dense Retrieval Encoders 46 Optimal Re-Ranking Depth (2026) Query Performance Prediction |
|
| 25 | Gustavo Penha | 44 | 3 | 58 Curriculum Learning Strategies for IR (2020) Ranking Distillation and Adaptation 52 Online learning to rank for sequential music recommendation (2019) Search and Recommendation Integration |
|
| 27 | Antonio Mallia | 44 | 3 | 46 Unveiling DIME: Reproducibility, Generalizability, and Formal Analysis of Dimension Importance Estimation for Dense Retrieval (2025) Query Reformulation and Expansion 45 Learning Passage Impacts for Inverted Indexes (2021) Learned Sparse Retrieval Efficiency |
|
| 28 | Tingyu Song | 44 | 3 | 63 Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems (2026) Reasoning-Intensive Retrieval and Ranking 59 LimRank: Less is More for Reasoning-Intensive Information Reranking (2025) Evidence Selection and Context Refinement |
|
| 29 | Jialong Wu 0007 | 44 | 3 | 42 AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptation (2025) Query Reformulation and Expansion 42 AdaCQR: Enhancing Query Reformulation for Conversational Search via Sparse and Dense Retrieval Alignment (2025) Query Reformulation and Expansion |
|
| 30 | Franco Maria Nardini | 43 | 3 | 46 Expansion via Prediction of Importance with Contextualization (2020) Sparse and Dense Retrieval Encoders 45 Learning Early Exit Strategies for Additive Ranking Ensembles (2021) Learned Sparse Retrieval Efficiency |
|
| 35 | Armin Toroghi | 43 | 3 | 55 Self-supervised Contrastive BERT Fine-tuning for Fusion-Based Reviewed-Item Retrieval (2023) Dense Bi-Encoder Training and Representation Learning 40 LogicRec: Recommendation with Users' Logical Requirements (2023) Knowledge Graph Embedding and Completion |
|
| 36 | Shu Zhou 0002 | 42 | 3 | 66 SCORE-RAG: Self-Correcting Exploration-Exploitation Retrieval for Multi-hop Question Answering (2026) Multi-Hop and Iterative Retrieval Reasoning 51 Why Knowledge Distillation Fails to Scale in Neural Retrieval (2026) Text Embedding and Representation Learning |
|
| 37 | Bowen Zheng 0005 COI: co-author | 42 | 3 | 55 Autonomous Reasoning-Retrieval for Large Language Model Based Recommendation (2025) Reasoning-Intensive Retrieval and Ranking 41 Generative Recommender with End-to-End Learnable Item Tokenization (2025) Item Tokenization and Semantic IDs |