Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests

sigir26-103 · Full research paper · Jingjie Ning, João Coelho, Yibo Kong, Yunfan Long, Bruno Martins, João Magalhães, Jamie Callan, Chenyan Xiong
User Behavior and Search Interaction · 50% Deep Research and Report Generation · 50%
Abstract

LLM-powered search agents are increasingly being used for multi-step information seeking tasks, yet the IR community lacks empirical understanding of how agentic search sessions unfold and how retrieved evidence is reflected in later queries. This paper presents a large-scale log analysis of agentic search based on 14.44M search requests (3.97M sessions) collected from DeepResearchGym, i.e., an open-source search API accessed by external agentic clients. We sessionize the logs, assign session-level intents and step-wise query-reformulation labels using LLM-based annotation, and propose Context-driven Term Adoption Rate (CTAR) to quantify whether newly introduced query terms are lexically traceable to previously retrieved evidence. Our analyses reveal distinctive behavioral patterns. First, over 90\% of multi-turn sessions contain at most ten steps, and 89\% of inter-step intervals fall under one minute. Second, behavior varies by intent. Fact-seeking sessions exhibit high repetition that increases over time, while sessions requiring reasoning sustain broader exploration. Third, query reformulations are often traceable to retrieved evidence across steps. On average, 54\% of newly introduced query terms appear in the accumulated evidence context, with additional traceability to earlier steps beyond the most recent retrieval. These findings provide candidate signals for repetition-aware stopping, intent-adaptive retrieval budgeting, and explicit cross-step context tracking. We released the anonymized logs, making them available at a public HuggingFace~\chref{https://huggingface.co/datasets/cx-cmu/deepresearchgym-agentic-search-logs}{repository}.

Assigned reviewers

#ReviewerMatchWhy (closest evidence publications in the taxonomy)
1 Sara Allawati
68
69 A Comparative Analysis of Linguistic and Retrieval Diversity in LLM-Generated Search Queries (2025) User Behavior and Search Interaction
64 What do Users Really Ask Large Language Models? An Initial Log Analysis of Google Bard Interactions in the Wild (2024) User Behavior and Search Interaction
64 From Lab to Reality: An Eye-tracking Study of How Users are Influenced to Search in the Era of GenAI (2026) User Behavior and Search Interaction
2 Romain Deffayet
50
64 Evaluating the Robustness of Click Models to Policy Distributional Shift (2023) User Behavior and Search Interaction
64 An Offline Metric for the Debiasedness of Click Models (2023) User Behavior and Search Interaction
34 SARDINE: Simulator for Automated Recommendation in Dynamic and Interactive Environments (2024) Recommendation Systems and Evaluation

Area chair

Area chairMatch
Ziyi Ye 54
Area-chair candidates (7)
Ziyi Ye assigned 5414 papers
Zhijing Wu 0001 449 papers
Xiaoxi Li 0005 3815 papers
Fernando Diaz 0001 COI: co-author 3512 papers
Jun Xu 0001 332 papers
Chenyan Xiong COI: author 3315 papers
Hongjin Qian 3212 papers

Candidates from the panel ranked by match score

#ReviewerMatchLoadWhy
1 Sara Allawati assigned 68 3
69 A Comparative Analysis of Linguistic and Retrieval Diversity in LLM-Generated Search Queries (2025) User Behavior and Search Interaction
64 What do Users Really Ask Large Language Models? An Initial Log Analysis of Google Bard Interactions in the Wild (2024) User Behavior and Search Interaction
2 Jiqun Liu 66 3
64 Characterizing the Stages of Complex Tasks (2019) User Behavior and Search Interaction
64 State-Aware Meta-Evaluation of Evaluation Metrics in Interactive Information Retrieval (2021) User Behavior and Search Interaction
3 Vuong Thanh Tung 66 3
68 Does More Context Help? Effects of Context Window and Application Source on Retrieval Performance (2022) User Behavior and Search Interaction
68 Predicting Representations of Information Needs from Digital Activity Context (2024) User Behavior and Search Interaction
4 Fan Zhang 0053 63 3
68 Incorporating Query Reformulating Behavior into Web Search Evaluation (2021) User Behavior and Search Interaction
68 Towards a Better Understanding of Query Reformulation Behavior in Web Search (2021) User Behavior and Search Interaction
6 Jaime Arguello COI: co-author 51 3
68 The Effects of Vertical Rank and Border on Aggregated Search Coherence and Search Behavior (2014) User Behavior and Search Interaction
68 The effect of aggregated search coherence on search behavior (2012) User Behavior and Search Interaction
7 Romain Deffayet assigned 50 3
64 Evaluating the Robustness of Click Models to Policy Distributional Shift (2023) User Behavior and Search Interaction
64 An Offline Metric for the Debiasedness of Click Models (2023) User Behavior and Search Interaction
8 Mark D. Smucker 48 3
68 Evaluating Streams of Evolving News Events (2015) User Behavior and Search Interaction
68 IR Evaluation: Modeling User Behavior for Measuring Effectiveness (2015) User Behavior and Search Interaction
9 Falk Scholer 48 3
70 On the Use of LLMs for Relevance Labelling (2026) User Behavior and Search Interaction
70 Enhancing Human Annotation: Leveraging Large Language Models and Efficient Batch Processing (2024) User Behavior and Search Interaction
10 Jialong Wu 0007 46 3
64 OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking (2025) Deep Research and Report Generation
64 WebDancer: Towards Autonomous Information Seeking Agency (2025) Deep Research and Report Generation
11 Paul Thomas 0001 44 3
70 Large Language Models can Accurately Predict Searcher Preferences (2024) User Behavior and Search Interaction
68 User Models, Metrics and Measures of Search: A Tutorial on the C/W/L Evaluation Framework (2021) User Behavior and Search Interaction
13 Xuanhui Wang 43 3
66 Learning with Sparse and Biased Feedback for Personal Search (2018) User Behavior and Search Interaction
66 Revisiting Two-tower Models for Unbiased Learning to Rank (2022) User Behavior and Search Interaction
14 Saber Zerhoudi 43 3
67 In-Browser Agents for Search Assistance (2026) User Behavior and Search Interaction
66 Beyond the Click: A Framework for Inferring Cognitive Traces in Search (2026) User Behavior and Search Interaction
15 Leif Azzopardi 39 3
68 User Models, Metrics and Measures of Search: A Tutorial on the C/W/L Evaluation Framework (2021) User Behavior and Search Interaction
68 Query quality: user ratings and system predictions (2010) User Behavior and Search Interaction
16 Wenlin Zhang 0001 39 3
63 Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery (2026) Deep Research and Report Generation
63 MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search (2026) Agentic and Deep Search RAG
17 Shuaiqiang Wang 39 3
70 M2oERank: Multi-Objective Mixture-of-Experts Enhanced Ranking for Satisfaction-Oriented Web Search (2025) User Behavior and Search Interaction
69 FULTR: A Large-Scale Fusion Learning to Rank Dataset and Its Application for Satisfaction-Oriented Ranking (2025) User Behavior and Search Interaction
19 Yujia Zhou 0002 37 3
69 Incorporating Explicit Subtopics in Personalized Search (2023) User Behavior and Search Interaction
69 Intent-Oriented Dynamic Interest Modeling for Personalized Web Search (2024) User Behavior and Search Interaction
20 Idan Szpektor COI: co-author 37 3
64 Predicting web searcher satisfaction with existing community-based answers (2011) User Behavior and Search Interaction
34 DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering (2023) Knowledge Conflict and Faithfulness
21 Tetsuya Sakai 37 3
36 Diversified search evaluation: lessons from the NTCIR-9 INTENT task (2013) Evaluation Metrics and User Models
36 Summaries, ranked retrieval and sessions: a unified framework for information access evaluation (2013) Evaluation Metrics and User Models
22 Zhen Qin 0001 37 3
66 Cross-Positional Attention for Debiasing Clicks (2021) User Behavior and Search Interaction
66 Revisiting Two-tower Models for Unbiased Learning to Rank (2022) User Behavior and Search Interaction
24 Yingyi Zhang 0001 35 3
63 Personalized Deep Research: A User-Centric Framework, Dataset, and Hybrid Evaluation for Knowledge Discovery (2026) Deep Research and Report Generation
63 MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search (2026) Agentic and Deep Search RAG
25 Yong Li 0004 34 3
64 Towards an Inclusive Mobile Web: A Dataset and Framework for Focusability in UI Accessibility (2025) User Behavior and Search Interaction
36 ProBench: Benchmarking GUI Agents with Accurate Process Information (2026) Reasoning Evaluation and Benchmarks
26 Tingyu Song 34 3
67 IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in Expert-Domain Information Retrieval (2025) User Behavior and Search Interaction
32 Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems (2026) Reasoning-Intensive Retrieval and Ranking
27 Weiwei Sun 0001 34 3
40 Simulating User Satisfaction for the Evaluation of Task-oriented Dialogue Systems (2021) Conversational Search and Retrieval
38 Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learning (2025) Agentic and Deep Search RAG
28 Gustavo Penha 33 3
70 As It Was: Aligning LLM Search Evaluation with Historical User Preferences (2026) User Behavior and Search Interaction
34 Semantic IDs for Joint Generative Search and Recommendation (2025) Recommendation with Search and Ranking
30 Xiao Zhang 0034 33 3
36 UniSAR: Modeling User Transition Behaviors between Search and Recommendation (2024) Recommendation with Search and Ranking
34 GenSAR: Unifying Balanced Search and Recommendation with Generative Retrieval (2025) Recommendation with Search and Ranking
32 Maryam Amirizaniani 33 3
62 Representing Tasks with a Graph-Based Method for Supporting Users in Complex Search Tasks (2023) User Behavior and Search Interaction
37 Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering (2026) Personalized Question Answering and RAG
33 Wonduk Seo 33 3
36 Automated Visualization Code Synthesis via Multi-path Reasoning and Feedback-Driven Optimizations (2026) Reasoning Evaluation and Benchmarks
36 SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Science (2026) Multi-Agent and Tool-Use Orchestration
34 Yang Song 0008 32 3
72 Query-Less: Predicting Task Repetition for NextGen Proactive Search and Recommendation Engines (2016) User Behavior and Search Interaction
34 Making Transformer Decoders Better Differentiable Indexers (2025) Generative Retrieval
35 Negar Arabzadeh 32 3
37 Query Performance Prediction: From Ad-hoc to Conversational Search (2023) Query Performance Prediction
34 Can QPP Choose the Right Query variant? Evaluating Query Variant Selection for RAG Pipelines (2026) Evidence Selection and Context Refinement
37 Haritz Puerto 32 3
33 MetaQA: Combining Expert Agents for Multi-Skill Question Answering (2023) Retrieval-Augmented and Open-Domain QA
31 Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs (2024) Reward Modeling and Process Supervision