Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests
Abstract
LLM-powered search agents are increasingly being used for multi-step information seeking tasks, yet the IR community lacks empirical understanding of how agentic search sessions unfold and how retrieved evidence is reflected in later queries. This paper presents a large-scale log analysis of agentic search based on 14.44M search requests (3.97M sessions) collected from DeepResearchGym, i.e., an open-source search API accessed by external agentic clients. We sessionize the logs, assign session-level intents and step-wise query-reformulation labels using LLM-based annotation, and propose Context-driven Term Adoption Rate (CTAR) to quantify whether newly introduced query terms are lexically traceable to previously retrieved evidence. Our analyses reveal distinctive behavioral patterns. First, over 90\% of multi-turn sessions contain at most ten steps, and 89\% of inter-step intervals fall under one minute. Second, behavior varies by intent. Fact-seeking sessions exhibit high repetition that increases over time, while sessions requiring reasoning sustain broader exploration. Third, query reformulations are often traceable to retrieved evidence across steps. On average, 54\% of newly introduced query terms appear in the accumulated evidence context, with additional traceability to earlier steps beyond the most recent retrieval. These findings provide candidate signals for repetition-aware stopping, intent-adaptive retrieval budgeting, and explicit cross-step context tracking. We released the anonymized logs, making them available at a public HuggingFace~\chref{https://huggingface.co/datasets/cx-cmu/deepresearchgym-agentic-search-logs}{repository}.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Sara Allawati | 68 | 69 A Comparative Analysis of Linguistic and Retrieval Diversity in LLM-Generated Search Queries (2025) User Behavior and Search Interaction 64 What do Users Really Ask Large Language Models? An Initial Log Analysis of Google Bard Interactions in the Wild (2024) User Behavior and Search Interaction 64 From Lab to Reality: An Eye-tracking Study of How Users are Influenced to Search in the Era of GenAI (2026) User Behavior and Search Interaction |
|
| 2 | Romain Deffayet | 50 | 64 Evaluating the Robustness of Click Models to Policy Distributional Shift (2023) User Behavior and Search Interaction 64 An Offline Metric for the Debiasedness of Click Models (2023) User Behavior and Search Interaction 34 SARDINE: Simulator for Automated Recommendation in Dynamic and Interactive Environments (2024) Recommendation Systems and Evaluation |
Area chair
| Area chair | Match | |
|---|---|---|
| Ziyi Ye | 54 |
Area-chair candidates (7)
| Ziyi Ye assigned | 54 | 14 papers | |
| Zhijing Wu 0001 | 44 | 9 papers | |
| Xiaoxi Li 0005 | 38 | 15 papers | |
| Fernando Diaz 0001 COI: co-author | 35 | 12 papers | |
| Jun Xu 0001 | 33 | 2 papers | |
| Chenyan Xiong COI: author | 33 | 15 papers | |
| Hongjin Qian | 32 | 12 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Sara Allawati assigned | 68 | 3 | 69 A Comparative Analysis of Linguistic and Retrieval Diversity in LLM-Generated Search Queries (2025) User Behavior and Search Interaction 64 What do Users Really Ask Large Language Models? An Initial Log Analysis of Google Bard Interactions in the Wild (2024) User Behavior and Search Interaction |
|
| 2 | Jiqun Liu | 66 | 3 | 64 Characterizing the Stages of Complex Tasks (2019) User Behavior and Search Interaction 64 State-Aware Meta-Evaluation of Evaluation Metrics in Interactive Information Retrieval (2021) User Behavior and Search Interaction |
|
| 3 | Vuong Thanh Tung | 66 | 3 | 68 Does More Context Help? Effects of Context Window and Application Source on Retrieval Performance (2022) User Behavior and Search Interaction 68 Predicting Representations of Information Needs from Digital Activity Context (2024) User Behavior and Search Interaction |
|
| 4 | Fan Zhang 0053 | 63 | 3 | 68 Incorporating Query Reformulating Behavior into Web Search Evaluation (2021) User Behavior and Search Interaction 68 Towards a Better Understanding of Query Reformulation Behavior in Web Search (2021) User Behavior and Search Interaction |
|
| 6 | Jaime Arguello COI: co-author | 51 | 3 | 68 The Effects of Vertical Rank and Border on Aggregated Search Coherence and Search Behavior (2014) User Behavior and Search Interaction 68 The effect of aggregated search coherence on search behavior (2012) User Behavior and Search Interaction |
|
| 7 | Romain Deffayet assigned | 50 | 3 | 64 Evaluating the Robustness of Click Models to Policy Distributional Shift (2023) User Behavior and Search Interaction 64 An Offline Metric for the Debiasedness of Click Models (2023) User Behavior and Search Interaction |
|
| 8 | Mark D. Smucker | 48 | 3 | 68 Evaluating Streams of Evolving News Events (2015) User Behavior and Search Interaction 68 IR Evaluation: Modeling User Behavior for Measuring Effectiveness (2015) User Behavior and Search Interaction |
|
| 9 | Falk Scholer | 48 | 3 | 70 On the Use of LLMs for Relevance Labelling (2026) User Behavior and Search Interaction 70 Enhancing Human Annotation: Leveraging Large Language Models and Efficient Batch Processing (2024) User Behavior and Search Interaction |
|
| 10 | Jialong Wu 0007 | 46 | 3 | 64 OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking (2025) Deep Research and Report Generation 64 WebDancer: Towards Autonomous Information Seeking Agency (2025) Deep Research and Report Generation |
|
| 11 | Paul Thomas 0001 | 44 | 3 | 70 Large Language Models can Accurately Predict Searcher Preferences (2024) User Behavior and Search Interaction 68 User Models, Metrics and Measures of Search: A Tutorial on the C/W/L Evaluation Framework (2021) User Behavior and Search Interaction |
|
| 13 | Xuanhui Wang | 43 | 3 | 66 Learning with Sparse and Biased Feedback for Personal Search (2018) User Behavior and Search Interaction 66 Revisiting Two-tower Models for Unbiased Learning to Rank (2022) User Behavior and Search Interaction |
|
| 14 | Saber Zerhoudi | 43 | 3 | 67 In-Browser Agents for Search Assistance (2026) User Behavior and Search Interaction 66 Beyond the Click: A Framework for Inferring Cognitive Traces in Search (2026) User Behavior and Search Interaction |
|
| 15 | Leif Azzopardi | 39 | 3 | 68 User Models, Metrics and Measures of Search: A Tutorial on the C/W/L Evaluation Framework (2021) User Behavior and Search Interaction 68 Query quality: user ratings and system predictions (2010) User Behavior and Search Interaction |
|
| 16 | Wenlin Zhang 0001 | 39 | 3 | 63 Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery (2026) Deep Research and Report Generation 63 MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search (2026) Agentic and Deep Search RAG |
|
| 17 | Shuaiqiang Wang | 39 | 3 | 70 M2oERank: Multi-Objective Mixture-of-Experts Enhanced Ranking for Satisfaction-Oriented Web Search (2025) User Behavior and Search Interaction 69 FULTR: A Large-Scale Fusion Learning to Rank Dataset and Its Application for Satisfaction-Oriented Ranking (2025) User Behavior and Search Interaction |
|
| 19 | Yujia Zhou 0002 | 37 | 3 | 69 Incorporating Explicit Subtopics in Personalized Search (2023) User Behavior and Search Interaction 69 Intent-Oriented Dynamic Interest Modeling for Personalized Web Search (2024) User Behavior and Search Interaction |
|
| 20 | Idan Szpektor COI: co-author | 37 | 3 | 64 Predicting web searcher satisfaction with existing community-based answers (2011) User Behavior and Search Interaction 34 DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering (2023) Knowledge Conflict and Faithfulness |
|
| 21 | Tetsuya Sakai | 37 | 3 | 36 Diversified search evaluation: lessons from the NTCIR-9 INTENT task (2013) Evaluation Metrics and User Models 36 Summaries, ranked retrieval and sessions: a unified framework for information access evaluation (2013) Evaluation Metrics and User Models |
|
| 22 | Zhen Qin 0001 | 37 | 3 | 66 Cross-Positional Attention for Debiasing Clicks (2021) User Behavior and Search Interaction 66 Revisiting Two-tower Models for Unbiased Learning to Rank (2022) User Behavior and Search Interaction |
|
| 24 | Yingyi Zhang 0001 | 35 | 3 | 63 Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery (2026) Deep Research and Report Generation 63 MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search (2026) Agentic and Deep Search RAG |
|
| 25 | Yong Li 0004 | 34 | 3 | 64 Towards an Inclusive Mobile Web: A Dataset and Framework for Focusability in UI Accessibility (2025) User Behavior and Search Interaction 36 ProBench: Benchmarking GUI Agents with Accurate Process Information (2026) Reasoning Evaluation and Benchmarks |
|
| 26 | Tingyu Song | 34 | 3 | 67 IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in Expert-Domain Information Retrieval (2025) User Behavior and Search Interaction 32 Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems (2026) Reasoning-Intensive Retrieval and Ranking |
|
| 27 | Weiwei Sun 0001 | 34 | 3 | 40 Simulating User Satisfaction for the Evaluation of Task-oriented Dialogue Systems (2021) Conversational Search and Retrieval 38 Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learning (2025) Agentic and Deep Search RAG |
|
| 28 | Gustavo Penha | 33 | 3 | 70 As It Was: Aligning LLM Search Evaluation with Historical User Preferences (2026) User Behavior and Search Interaction 34 Semantic IDs for Joint Generative Search and Recommendation (2025) Recommendation with Search and Ranking |
|
| 30 | Xiao Zhang 0034 | 33 | 3 | 36 UniSAR: Modeling User Transition Behaviors between Search and Recommendation (2024) Recommendation with Search and Ranking 34 GenSAR: Unifying Balanced Search and Recommendation with Generative Retrieval (2025) Recommendation with Search and Ranking |
|
| 32 | Maryam Amirizaniani | 33 | 3 | 62 Representing Tasks with a Graph-Based Method for Supporting Users in Complex Search Tasks (2023) User Behavior and Search Interaction 37 Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering (2026) Personalized Question Answering and RAG |
|
| 33 | Wonduk Seo | 33 | 3 | 36 Automated Visualization Code Synthesis via Multi-path Reasoning and Feedback-Driven Optimizations (2026) Reasoning Evaluation and Benchmarks 36 SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Science (2026) Multi-Agent and Tool-Use Orchestration |
|
| 34 | Yang Song 0008 | 32 | 3 | 72 Query-Less: Predicting Task Repetition for NextGen Proactive Search and Recommendation Engines (2016) User Behavior and Search Interaction 34 Making Transformer Decoders Better Differentiable Indexers (2025) Generative Retrieval |
|
| 35 | Negar Arabzadeh | 32 | 3 | 37 Query Performance Prediction: From Ad-hoc to Conversational Search (2023) Query Performance Prediction 34 Can QPP Choose the Right Query variant? Evaluating Query Variant Selection for RAG Pipelines (2026) Evidence Selection and Context Refinement |
|
| 37 | Haritz Puerto | 32 | 3 | 33 MetaQA: Combining Expert Agents for Multi-Skill Question Answering (2023) Retrieval-Augmented and Open-Domain QA 31 Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs (2024) Reward Modeling and Process Supervision |