A Comparative Study of Users' Information-Seeking Practices Across Search Engines and Generative AI Chatbots
Abstract
Web search engines have long been the primary gateway to online information, but generative AI chatbots are emerging as alternative tools. Yet we lack empirical understanding of how users choose between these tools and how their information-seeking behaviors differ across these contexts. Understanding how users select, use, and assess information retrieval (IR) systems is essential for evaluating them in ways that reflect user expectations across contexts. To shed light on user perceptions and usage of IR systems, we surveyed 84 participants about their recent search engine and GenAI chatbot use, analyzing task types, contextual triggers, outcomes, and tool preferences. Our findings reveal that users approach search engines and GenAI chatbots with distinct philosophies. With search engines, they follow a 'Browse and Verify' approach, prioritizing reliability and source verifiability. With GenAI chatbots, they adopt a 'Delegate and Consume' approach, valuing quick summarization and personalized content. Despite these stated preferences, actual usage shows that users frequently turn to GenAI chatbots for actionable guidance in high-stakes domains such as health and relationship matters, hinting at a divergence between stated philosophies and real-world behavior. These findings underscore the need for IR systems that support reliable information assessment, transparent sources, and user-guided evaluation.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Saber Zerhoudi | 47 | 100 From SERPs to Agents: A Platform for Comparative Studies of Information Interaction (2026) User Behavior and Search Interaction 88 Beyond the Click: A Framework for Inferring Cognitive Traces in Search (2026) User Behavior and Search Interaction 85 In-Browser Agents for Search Assistance (2026) User Behavior and Search Interaction |
|
| 2 | Leif Azzopardi | 47 | 72 TREC iKAT 2023: A Test Collection for Evaluating Conversational and Interactive Knowledge Assistants (2024) Interactive Information Seeking and Evaluation 72 Second International Workshop On the Evaluation of Collaborative Information Seeking and Retrieval (Ecol'17) (2017) Interactive Information Seeking and Evaluation 72 User Models, Metrics and Measures of Search: A Tutorial on the C/W/L Evaluation Framework (2021) User Behavior and Search Interaction |
Area chair
| Area chair | Match | |
|---|---|---|
| Ziyi Ye | 51 |
Area-chair candidates (8)
| Ziyi Ye assigned | 51 | 14 papers | |
| Zhijing Wu 0001 | 40 | 9 papers | |
| Fernando Diaz 0001 | 32 | 12 papers | |
| Jun Xu 0001 | 30 | 2 papers | |
| Chenyan Xiong | 29 | 15 papers | |
| Kam-Fai Wong | 29 | 6 papers | |
| Xiaoxi Li 0005 | 28 | 15 papers | |
| Xiaoxue Zang | 28 | 4 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Vuong Thanh Tung | 75 | 3 | 85 Spoken Conversational Context Improves Query Auto-completion in Web Search (2021) User Behavior and Search Interaction 72 Understanding user behavior in naturalistic information search tasks (2019) User Behavior and Search Interaction |
|
| 2 | Jiqun Liu | 74 | 3 | 72 Task Supportive and Personalized Human-Large Language Model Interaction: A User Study (2024) Interactive Information Seeking and Evaluation 72 Characterizing the Stages of Complex Tasks (2019) User Behavior and Search Interaction |
|
| 3 | Sara Allawati | 68 | 3 | 72 From Lab to Reality: An Eye-tracking Study of How Users are Influenced to Search in the Era of GenAI (2026) User Behavior and Search Interaction 72 What do Users Really Ask Large Language Models? An Initial Log Analysis of Google Bard Interactions in the Wild (2024) User Behavior and Search Interaction |
|
| 4 | Fan Zhang 0053 | 67 | 3 | 100 How Users Interact with Generative Information Retrieval Systems: A Study of User Behavior and Search Experience (2025) User Behavior and Search Interaction 100 Conversational vs Traditional: Comparing Search Behavior and Outcome in Legal Case Retrieval (2021) User Behavior and Search Interaction |
|
| 5 | Jaime Arguello | 55 | 3 | 100 The effects of goal-setting on learning during information seeking with generative AI (2026) User Behavior and Search Interaction 72 Goal-setting in support of learning during search: An exploration of learning outcomes and searcher perceptions (2023) User Behavior and Search Interaction |
|
| 7 | Saber Zerhoudi assigned | 47 | 3 | 100 From SERPs to Agents: A Platform for Comparative Studies of Information Interaction (2026) User Behavior and Search Interaction 88 Beyond the Click: A Framework for Inferring Cognitive Traces in Search (2026) User Behavior and Search Interaction |
|
| 8 | Mark D. Smucker | 47 | 3 | 72 IR Evaluation: Modeling User Behavior for Measuring Effectiveness (2015) User Behavior and Search Interaction 72 Time-based calibration of effectiveness measures (2012) User Behavior and Search Interaction |
|
| 9 | Leif Azzopardi assigned | 47 | 3 | 72 TREC iKAT 2023: A Test Collection for Evaluating Conversational and Interactive Knowledge Assistants (2024) Interactive Information Seeking and Evaluation 72 Second International Workshop On the Evaluation of Collaborative Information Seeking and Retrieval (Ecol'17) (2017) Interactive Information Seeking and Evaluation |
|
| 10 | Romain Deffayet | 46 | 3 | 72 Evaluating the Robustness of Click Models to Policy Distributional Shift (2023) User Behavior and Search Interaction 72 An Offline Metric for the Debiasedness of Click Models (2023) User Behavior and Search Interaction |
|
| 11 | Paul Thomas 0001 | 44 | 3 | 72 Focused and aggregated search: a perspective from natural language generation (2010) Interactive Information Seeking and Evaluation 72 Third Workshop on Building towards Information Interaction and Retrieval Resources Re-use (BIIRRR 2022) (2022) User Behavior and Search Interaction |
|
| 12 | Falk Scholer | 44 | 3 | 70 Classifying Term Variants in Query Formulation (2025) User Behavior and Search Interaction 70 Can Users Predict Relative Query Effectiveness? (2022) User Behavior and Search Interaction |
|
| 13 | Xuanhui Wang | 41 | 3 | 72 Cross-Positional Attention for Debiasing Clicks (2021) User Behavior and Search Interaction 65 Learning with Sparse and Biased Feedback for Personal Search (2018) User Behavior and Search Interaction |
|
| 15 | Tetsuya Sakai | 35 | 3 | 40 Diversified search evaluation: lessons from the NTCIR-9 INTENT task (2013) Evaluation Metrics and User Models 40 Summaries, ranked retrieval and sessions: a unified framework for information access evaluation (2013) Evaluation Metrics and User Models |
|
| 16 | Zhen Qin 0001 | 33 | 3 | 65 Cross-Positional Attention for Debiasing Clicks (2021) User Behavior and Search Interaction 65 Revisiting Two-tower Models for Unbiased Learning to Rank (2022) User Behavior and Search Interaction |
|
| 17 | Shuaiqiang Wang | 33 | 3 | 66 FULTR: A Large-Scale Fusion Learning to Rank Dataset and Its Application for Satisfaction-Oriented Ranking (2025) User Behavior and Search Interaction 64 RankExpert: A Mixture of Textual-and-Behavioral Experts for Multi-Objective Learning-to-Rank in Web Search (2025) User Behavior and Search Interaction |
|
| 18 | Yujia Zhou 0002 | 33 | 3 | 69 Intent-Oriented Dynamic Interest Modeling for Personalized Web Search (2024) User Behavior and Search Interaction 69 Incorporating Explicit Subtopics in Personalized Search (2023) User Behavior and Search Interaction |
|
| 19 | Negar Arabzadeh | 32 | 3 | 50 Query Performance Prediction: From Ad-hoc to Conversational Search (2023) Query Performance Prediction 34 Matches Made in Heaven: Toolkit and Large-Scale Datasets for Supervised Query Reformulation (2021) Query Reformulation and Expansion |
|
| 21 | Yifan Wang 0017 | 32 | 3 | 100 User Perception of Recommendation Explanation: Are Your Explanations What Users Need? (2023) User Behavior and Search Interaction 26 Improving Long-tail User CTR Prediction via Hierarchical Distribution Alignment (2025) Bias Mitigation Methods |
|
| 22 | Idan Szpektor | 32 | 3 | 72 Predicting web searcher satisfaction with existing community-based answers (2011) User Behavior and Search Interaction 36 Identifying Web Queries with Question Intent (2016) Personalized and Context-Aware Retrieval |
|
| 23 | Gustavo Penha | 31 | 3 | 67 As It Was: Aligning LLM Search Evaluation with Historical User Preferences (2026) User Behavior and Search Interaction 34 Contextualizing Spotify's Audiobook List Recommendations with Descriptive Shelves (2025) Recommendation Systems and Evaluation |
|
| 24 | Theresia Veronika Rampisela | 30 | 3 | 30 Measuring Individual User Fairness with User Similarity and Effectiveness Disparity (2026) Fairness Metrics and Evaluation 30 Relevance-aware Individual Item Fairness Measures for Recommender Systems: Limitations and Usage Guidelines (2026) Fairness Metrics and Evaluation |
|
| 26 | Yong Li 0004 | 30 | 3 | 72 Towards an Inclusive Mobile Web: A Dataset and Framework for Focusability in UI Accessibility (2025) User Behavior and Search Interaction 31 Efficient Memory Alignment for Long-term Conversational Information Seeking (2026) Conversational Search and Retrieval |
|
| 27 | Sunhao Dai | 29 | 3 | 100 NExT-Search: Rebuilding User Feedback Ecosystem for Generative AI Search (2025) User Behavior and Search Interaction 30 AdaO2B: Adaptive Online to Batch Conversion for Out-of-Distribution Generalization (2025) Long-Term Engagement and Retention Optimization |
|
| 28 | Jialong Wu 0007 | 29 | 3 | 33 AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptation (2025) Query Reformulation and Expansion 33 AdaCQR: Enhancing Query Reformulation for Conversational Search via Sparse and Dense Retrieval Alignment (2025) Query Reformulation and Expansion |
|
| 29 | Xiao Zhang 0034 | 29 | 3 | 32 UniSAR: Modeling User Transition Behaviors between Search and Recommendation (2024) Recommendation with Search and Ranking 32 Similarity = Value? Consultation Value-Assessment and Alignment for Personalized Search (2025) Personalized and Context-Aware Retrieval |
|
| 30 | Zechun Niu | 29 | 3 | 33 Investigating the Robustness of Counterfactual Learning to Rank Models: A Reproducibility Study (2025) Unbiased Learning to Rank 28 Addressing Personalized Bias for Unbiased Learning to Rank (2025) Unbiased Learning to Rank |
|
| 33 | Yang Song 0008 | 29 | 3 | 68 Query-Less: Predicting Task Repetition for NextGen Proactive Search and Recommendation Engines (2016) User Behavior and Search Interaction 34 LabelCraft: Empowering Short Video Recommendations with Automated Label Crafting (2024) Recommendation Systems and Evaluation |
|
| 34 | Bo Peng 0009 | 29 | 3 | 65 SWaT: Statistical Modeling of Video Watch Time through User Behavior Analysis (2025) User Behavior and Search Interaction 27 eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data (2024) Efficiency and Deployment |
|
| 35 | Tingyu Song | 29 | 3 | 65 IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in Expert-Domain Information Retrieval (2025) User Behavior and Search Interaction 25 Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems (2026) Reasoning-Intensive Retrieval and Ranking |
|
| 37 | Yuta Saito | 28 | 3 | 34 Evaluating the Robustness of Off-Policy Evaluation (2021) Recommendation Systems and Evaluation 33 Off-Policy Evaluation of Ranking Policies under Diverse User Behavior (2023) Recommendation with Search and Ranking |