Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Following the Eye-Tracking Evidence: Established Web-Search Assumptions Fail in Carousel Interfaces

sigir26-133 · Full research paper · Jingwei Kang, Maarten de Rijke, Harrie Oosterhuis
User Behavior and Search Interaction · 100%
Abstract

Carousel interfaces have been the de-facto standard for streaming media services for over a decade. Yet, there has been very little research into user behavior with such interfaces, which thus remains poorly understood. Due to this lack of empirical research, previous work has assumed that behaviors established in single-list web-search interfaces, such as the F-pattern and the examination hypothesis, also apply to carousel interfaces, for instance when designing click models or evaluation metrics. We analyze a recently-released interaction and examination dataset resulting from an eye-tracking study performed on carousel interfaces to verify whether these assumptions actually hold. We find that (i)~the F-pattern holds only for vertical examination and not for horizontal swiping; additionally, we discover that, when conditioned on a click, user examination follows an L-pattern unique to carousel interfaces; (ii)~click-through-rates conditioned on examination indicate that the well-known examination hypothesis does not hold in carousel interfaces; and (iii)~contrary to the assumptions of previous work, users generally ignore carousel headings and focus directly on the content items. Our findings show that many user behavior assumptions, especially concerning examination patterns, do not transfer from web search interfaces to carousel recommendation settings. Our work shows that the field lacks a reliable foundation on which to build models of user behavior with these interfaces. Consequently, a re-evaluation of existing metrics and click models for carousel interfaces may be warranted.

Assigned reviewers

#ReviewerMatchWhy (closest evidence publications in the taxonomy)
1 Jiqun Liu
81
100 The Paradox of Personalization: Does Task Prediction Require Individualized Models? (2018) User Behavior and Search Interaction
100 A Reference-Dependent Model for Web Search Evaluation: Understanding and Measuring the Experience of Boundedly Rational Users (2023) User Behavior and Search Interaction
100 Constructing and meta-evaluating state-aware evaluation metrics for interactive search systems (2023) User Behavior and Search Interaction
2 Fan Zhang 0053
59
100 Evaluating Web Search with a Bejeweled Player Model (2017) User Behavior and Search Interaction
100 Models Versus Satisfaction: Towards a Better Understanding of Evaluation Metrics (2020) User Behavior and Search Interaction
100 Constructing Better Evaluation Metrics by Incorporating the Anchoring Effect into the User Model (2022) User Behavior and Search Interaction

Area chair

Area chairMatch
Zhijing Wu 0001 34
Area-chair candidates (7)
Ziyi Ye COI: co-author 4514 papers
Zhijing Wu 0001 assigned 349 papers
Fernando Diaz 0001 COI: co-author 2712 papers
Xiaoxi Li 0005 2515 papers
Jun Xu 0001 COI: co-author 252 papers
Chenyan Xiong 2415 papers
Xiaoxue Zang 234 papers

Candidates from the panel ranked by match score

#ReviewerMatchLoadWhy
1 Jiqun Liu assigned 81 3
100 The Paradox of Personalization: Does Task Prediction Require Individualized Models? (2018) User Behavior and Search Interaction
100 A Reference-Dependent Model for Web Search Evaluation: Understanding and Measuring the Experience of Boundedly Rational Users (2023) User Behavior and Search Interaction
2 Sara Allawati COI: co-author 72 3
100 From Lab to Reality: An Eye-tracking Study of How Users are Influenced to Search in the Era of GenAI (2026) User Behavior and Search Interaction
100 What do Users Really Ask Large Language Models? An Initial Log Analysis of Google Bard Interactions in the Wild (2024) User Behavior and Search Interaction
3 Vuong Thanh Tung 72 3
100 Associating cognitive abilities with naturalistic search behavior (2025) User Behavior and Search Interaction
100 Understanding user behavior in naturalistic information search tasks (2019) User Behavior and Search Interaction
4 Fan Zhang 0053 assigned 59 3
100 Evaluating Web Search with a Bejeweled Player Model (2017) User Behavior and Search Interaction
100 Models Versus Satisfaction: Towards a Better Understanding of Evaluation Metrics (2020) User Behavior and Search Interaction
5 Jaime Arguello 47 3
100 Goal-setting in support of learning during search: An exploration of learning outcomes and searcher perceptions (2023) User Behavior and Search Interaction
100 Learning assessments in search-as-learning: A survey of prior work and opportunities for future research (2022) User Behavior and Search Interaction
7 Romain Deffayet COI: co-author 43 3
100 Evaluating the Robustness of Click Models to Policy Distributional Shift (2023) User Behavior and Search Interaction
100 An Offline Metric for the Debiasedness of Click Models (2023) User Behavior and Search Interaction
8 Mark D. Smucker 39 3
68 Time-based calibration of effectiveness measures (2012) User Behavior and Search Interaction
68 IR Evaluation: Modeling User Behavior for Measuring Effectiveness (2015) User Behavior and Search Interaction
9 Falk Scholer 39 3
64 Can Users Predict Relative Query Effectiveness? (2022) User Behavior and Search Interaction
64 Classifying Term Variants in Query Formulation (2025) User Behavior and Search Interaction
10 Saber Zerhoudi 37 3
72 From SERPs to Agents: A Platform for Comparative Studies of Information Interaction (2026) User Behavior and Search Interaction
72 Beyond the Click: A Framework for Inferring Cognitive Traces in Search (2026) User Behavior and Search Interaction
11 Xuanhui Wang COI: co-author 36 3
100 Cross-Positional Attention for Debiasing Clicks (2021) User Behavior and Search Interaction
62 Revisiting Two-tower Models for Unbiased Learning to Rank (2022) User Behavior and Search Interaction
12 Paul Thomas 0001 36 3
100 Third Workshop on Building towards Information Interaction and Retrieval Resources Re-use (BIIRRR 2022) (2022) User Behavior and Search Interaction
68 User Models, Metrics and Measures of Search: A Tutorial on the C/W/L Evaluation Framework (2021) User Behavior and Search Interaction
14 Leif Azzopardi COI: co-author 33 3
68 User Models, Metrics and Measures of Search: A Tutorial on the C/W/L Evaluation Framework (2021) User Behavior and Search Interaction
64 Query quality: user ratings and system predictions (2010) User Behavior and Search Interaction
15 Tetsuya Sakai 30 3
36 Give me just one highly relevant document: P-measure (2006) Evaluation Metrics and User Models
36 Summaries, ranked retrieval and sessions: a unified framework for information access evaluation (2013) Evaluation Metrics and User Models
16 Shuaiqiang Wang COI: co-author 29 3
61 RankExpert: A Mixture of Textual-and-Behavioral Experts for Multi-Objective Learning-to-Rank in Web Search (2025) User Behavior and Search Interaction
61 M2oERank: Multi-Objective Mixture-of-Experts Enhanced Ranking for Satisfaction-Oriented Web Search (2025) User Behavior and Search Interaction
17 Yujia Zhou 0002 29 3
62 Intent-Oriented Dynamic Interest Modeling for Personalized Web Search (2024) User Behavior and Search Interaction
62 Incorporating Explicit Subtopics in Personalized Search (2023) User Behavior and Search Interaction
18 Zhen Qin 0001 COI: co-author 29 3
62 Cross-Positional Attention for Debiasing Clicks (2021) User Behavior and Search Interaction
62 Revisiting Two-tower Models for Unbiased Learning to Rank (2022) User Behavior and Search Interaction
20 Idan Szpektor 27 3
100 Predicting web searcher satisfaction with existing community-based answers (2011) User Behavior and Search Interaction
33 Identifying Web Queries with Question Intent (2016) Personalized and Context-Aware Retrieval
21 Yuta Saito COI: co-author 26 3
31 Long-term Off-Policy Evaluation and Learning (2024) Long-Term Engagement and Retention Optimization
31 Counterfactual Evaluation and Learning for Interactive Systems: Foundations, Implementations, and Recent Advances (2022) Long-Term Engagement and Retention Optimization
22 Gustavo Penha 26 3
62 As It Was: Aligning LLM Search Evaluation with Historical User Preferences (2026) User Behavior and Search Interaction
29 Contextualizing Spotify's Audiobook List Recommendations with Descriptive Shelves (2025) Recommendation Systems and Evaluation
23 Bo Peng 0009 26 3
65 SWaT: Statistical Modeling of Video Watch Time through User Behavior Analysis (2025) User Behavior and Search Interaction
23 eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data (2024) Efficiency and Deployment
24 Xiao Zhang 0034 26 3
31 Counterfactual Reward Modification for Streaming Recommendation with Delayed Feedback (2021) Long-Term Engagement and Retention Optimization
31 HyperBandit: Contextual Bandit with Hypernewtork for Time-Varying User Preferences in Streaming Recommendation (2023) Long-Term Engagement and Retention Optimization
25 Negar Arabzadeh COI: co-author 25 3
32 Query Performance Prediction: From Ad-hoc to Conversational Search (2023) Query Performance Prediction
28 Estimating Query Performance Through Rich Contextualized Query Representations (2024) Query Performance Prediction
26 Yifan Wang 0017 25 3
72 User Perception of Recommendation Explanation: Are Your Explanations What Users Need? (2023) User Behavior and Search Interaction
24 Unbiased Delayed Feedback Label Correction for Conversion Rate Prediction (2023) Advertising and Auction Mechanisms
27 Yong Li 0004 25 3
100 Towards an Inclusive Mobile Web: A Dataset and Framework for Focusability in UI Accessibility (2025) User Behavior and Search Interaction
26 ProBench: Benchmarking GUI Agents with Accurate Process Information (2026) Reasoning Evaluation and Benchmarks
30 Yang Song 0008 25 3
64 Query-Less: Predicting Task Repetition for NextGen Proactive Search and Recommendation Engines (2016) User Behavior and Search Interaction
30 Making Transformer Decoders Better Differentiable Indexers (2025) Generative Retrieval
31 Jialong Wu 0007 25 3
29 OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking (2025) Deep Research and Report Generation
28 Nested Browser-Use Learning for Agentic Information Seeking (2026) Deep Research and Report Generation
32 Tingyu Song 24 3
61 IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in Expert-Domain Information Retrieval (2025) User Behavior and Search Interaction
20 Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems (2026) Reasoning-Intensive Retrieval and Ranking
33 Zechun Niu 24 3
26 Investigating the Robustness of Counterfactual Learning to Rank Models: A Reproducibility Study (2025) Unbiased Learning to Rank
24 Addressing Personalized Bias for Unbiased Learning to Rank (2025) Unbiased Learning to Rank
34 Sunhao Dai 24 3
72 NExT-Search: Rebuilding User Feedback Ecosystem for Generative AI Search (2025) User Behavior and Search Interaction
31 AdaO2B: Adaptive Online to Batch Conversion for Out-of-Distribution Generalization (2025) Long-Term Engagement and Retention Optimization
35 Theresia Veronika Rampisela 24 3
24 Relevance-aware Individual Item Fairness Measures for Recommender Systems: Limitations and Usage Guidelines (2026) Fairness Metrics and Evaluation
24 Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study (2025) Fairness Metrics and Evaluation