Equity vs. Equality: Optimizing Ranking Fairness for Tailored Provider Needs
sigir26-123 · Full research paper · Yiteng Tu, Weihang Su, Shuguang Han, Yiqun Liu, Qingyao Ai
Fairness Metrics and Evaluation · 50% Evaluation Metrics and User Models · 50%
Abstract
Ranking plays a central role in connecting users and providers in Information Retrieval (IR) systems, making provider-side fairness an important challenge. While recent research has begun to address fairness in ranking, most existing approaches adopt an equality-based perspective, aiming to ensure that providers with similar content receive similar exposure. However, it overlooks the diverse needs of real-world providers, whose utility from ranking may depend not only on exposure but also on outcomes like sales or engagement. Consequently, exposure-based fairness may not accurately capture the true utility perceived by different providers with varying priorities. To this end, we introduce an equity-oriented fairness framework that explicitly models each provider's preferences over key outcomes such as exposure and sales, thus evaluating whether a ranking algorithm can fulfill these individualized goals while maintaining overall fairness across providers. Based on this framework, we develop EquityRank, a gradient-based algorithm that jointly optimizes user-side effectiveness and provider-side equity. Extensive offline and online simulations demonstrate that EquityRank offers improved trade-offs between effectiveness and fairness and adapts to heterogeneous provider needs.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Theresia Veronika Rampisela | 70 | 70 Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study (2025) Fairness Metrics and Evaluation 70 Can We Trust Recommender System Fairness Evaluation? The Role of Fairness and Relevance (2024) Fairness Metrics and Evaluation 70 Relevance-aware Individual Item Fairness Measures for Recommender Systems: Limitations and Usage Guidelines (2026) Fairness Metrics and Evaluation |
|
| 2 | Sruthi Gorantla | 62 | 70 Sampling Individually-Fair Rankings that are Always Group Fair (2023) Fairness Metrics and Evaluation 70 Optimizing Learning-to-Rank Models for Ex-Post Fair Relevance (2024) Fairness Metrics and Evaluation 70 On the Problem of Underranking in Group-Fair Ranking (2021) Fairness Metrics and Evaluation |
|
| 3 | Leif Azzopardi | 50 | 100 Measuring Bias in a Ranked List Using Term-Based Representations (2024) Evaluation Metrics and User Models 100 Efficiently Estimating Retrievability Bias (2014) Evaluation Metrics and User Models 74 Retrievability Bias Estimation Using Synthetically Generated Queries (2023) Test Collection Construction and Pooling |
Area chair
| Area chair | Match | |
|---|---|---|
| Fernando Diaz 0001 | 45 |
Area-chair candidates (3)
| Fernando Diaz 0001 assigned | 45 | 12 papers | |
| Xiaoxue Zang | 37 | 4 papers | |
| Jun Xu 0001 COI: co-author | 36 | 2 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Theresia Veronika Rampisela assigned | 70 | 3 | 70 Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study (2025) Fairness Metrics and Evaluation 70 Can We Trust Recommender System Fairness Evaluation? The Role of Fairness and Relevance (2024) Fairness Metrics and Evaluation |
|
| 2 | Tetsuya Sakai COI: co-author | 64 | 3 | 70 Diversification as Risk Minimization (2026) Evaluation Metrics and User Models 70 Summaries, ranked retrieval and sessions: a unified framework for information access evaluation (2013) Evaluation Metrics and User Models |
|
| 3 | Sruthi Gorantla assigned | 62 | 3 | 70 Sampling Individually-Fair Rankings that are Always Group Fair (2023) Fairness Metrics and Evaluation 70 Optimizing Learning-to-Rank Models for Ex-Post Fair Relevance (2024) Fairness Metrics and Evaluation |
|
| 4 | Mark D. Smucker | 51 | 3 | 70 Offline Evaluation by Maximum Similarity to an Ideal Ranking (2020) Evaluation Metrics and User Models 70 Assessing Top- Preferences (2021) Evaluation Metrics and User Models |
|
| 5 | Leif Azzopardi assigned | 50 | 3 | 100 Measuring Bias in a Ranked List Using Term-Based Representations (2024) Evaluation Metrics and User Models 100 Efficiently Estimating Retrievability Bias (2014) Evaluation Metrics and User Models |
|
| 6 | Zechun Niu COI: co-author | 47 | 3 | 55 Addressing Personalized Bias for Unbiased Learning to Rank (2025) Unbiased Learning to Rank 45 Investigating the Robustness of Counterfactual Learning to Rank Models: A Reproducibility Study (2025) Unbiased Learning to Rank |
|
| 7 | Yimeng Bai | 45 | 3 | 62 Recommendation Unlearning via Influence Function (2025) Efficient Training and Data Optimization 43 Leveraging Watch-time Feedback for Short-Video Recommendations: A Causal Labeling Framework (2023) Recommendation Systems and Evaluation |
|
| 9 | Paul Thomas 0001 | 44 | 3 | 73 On the Use of LLMs for Relevance Labelling (2026) LLM-Based Relevance Assessment 62 User Models, Metrics and Measures of Search: A Tutorial on the C/W/L Evaluation Framework (2021) User Behavior and Search Interaction |
|
| 10 | Andrew Parry | 43 | 3 | 73 Large Language Model Relevance Assessors Agree With One Another More Than With Human Assessors (2025) LLM-Based Relevance Assessment 45 [inline-graphic not available: see fulltext] SuiteEval: Simplifying Retrieval Benchmarks (2026) Retrieval Toolkits and Reproducibility |
|
| 11 | Negar Arabzadeh | 40 | 3 | 44 Matches Made in Heaven: Toolkit and Large-Scale Datasets for Supervised Query Reformulation (2021) Query Reformulation and Expansion 44 A Reproducibility Study of LLM-Based Query Reformulation (2026) Query Reformulation and Expansion |
|
| 12 | Yifan Wang 0017 COI: co-author | 40 | 3 | 62 A Survey on the Fairness of Recommender Systems (2023) Popularity Bias and Long-Tail Debiasing 39 Improving Long-tail User CTR Prediction via Hierarchical Distribution Alignment (2025) Bias Mitigation Methods |
|
| 13 | Haritz Puerto | 40 | 3 | 70 Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers (2025) Fairness Metrics and Evaluation 31 C-SEO Bench: Does Conversational SEO Work? (2025) Reasoning Evaluation and Benchmarks |
|
| 14 | Xiao Wang 0043 COI: co-author | 39 | 3 | 45 Retrieve, Rerank, Answer, Experiment: Hands-On IR Research with PyTerrier (2026) Retrieval Toolkits and Reproducibility 39 Effective Contrastive Weighting for Dense Query Expansion (2023) Query Reformulation and Expansion |
|
| 15 | Xuanhui Wang COI: co-author | 39 | 3 | 43 TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank (2019) Retrieval Toolkits and Reproducibility 43 Rax: Composable Learning-to-Rank Using JAX (2022) Retrieval Toolkits and Reproducibility |
|
| 16 | Jizhi Zhang | 39 | 3 | 54 Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation (2023) Preference Alignment and Optimization 42 On the Theories Behind Hard Negative Sampling for Recommendation (2023) Recommendation Systems and Evaluation |
|
| 17 | Min Hou 0004 | 39 | 3 | 70 Fair Personalized Learner Modeling Without Sensitive Attributes (2025) Fairness Metrics and Evaluation 39 RecCocktail: A Generalizable and Efficient Framework for LLM-Based Recommendation (2026) Preference Alignment and Optimization |
|
| 18 | Chongming Gao | 39 | 3 | 43 KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems (2022) Recommendation Systems and Evaluation 42 On the Theories Behind Hard Negative Sampling for Recommendation (2023) Recommendation Systems and Evaluation |
|
| 19 | Xialong Liu | 38 | 3 | 70 Fairness-Aware Design for Contextual Experiments: Guaranteeing Reliability and Equity in Heterogeneous Subgroups (2026) Fairness Metrics and Evaluation 42 CROLoss: Towards a Customizable Loss for Retrieval Models in Recommender Systems (2022) Recommendation with Search and Ranking |
|
| 20 | Jaime Arguello | 38 | 3 | 44 Multilingual and Domain-Agnostic Tip-of-the-Tongue Query Generation for Simulated Evaluation (2026) User Simulation for Interactive Systems 44 Tip of the Tongue Query Elicitation for Simulated Evaluation (2025) User Simulation for Interactive Systems |
|
| 21 | Yang Zhang 0072 | 38 | 3 | 69 Exact and Efficient Unlearning for Large Language Model-Based Recommendation (2025) Efficiency and Deployment 40 Preliminary Study on Incremental Learning for Large Language Model-based Recommender Systems (2024) Efficiency and Deployment |
|
| 22 | Zhen Qin 0001 | 38 | 3 | 43 Improving Recommendation Quality in Google Drive (2020) Recommendation Systems and Evaluation 43 Bootstrapping Recommendations at Chrome Web Store (2021) Recommendation Systems and Evaluation |
|
| 23 | Fan Zhang 0053 COI: co-author | 38 | 3 | 62 Evaluating Mobile Search with Height-Biased Gain (2017) User Behavior and Search Interaction 49 Incorporating Query Reformulating Behavior into Web Search Evaluation (2021) User Behavior and Search Interaction |
|
| 24 | Rolf Jagerman | 38 | 3 | 43 Bootstrapping Recommendations at Chrome Web Store (2021) Recommendation Systems and Evaluation 43 Rax: Composable Learning-to-Rank Using JAX (2022) Retrieval Toolkits and Reproducibility |
|
| 25 | Falk Scholer COI: co-author | 38 | 3 | 42 Can Generative LLMs Create Query Variants for Test Collections? An Exploratory Study (2023) Query Reformulation and Expansion 42 sMARE: a new paradigm to evaluate and understand query performance prediction methods (2022) Query Performance Prediction |
|
| 26 | Daiting Shi | 38 | 3 | 70 MA4DIV: Multi-Agent Reinforcement Learning for Search Result Diversification (2025) Evaluation Metrics and User Models 42 Population-Guided Intent-Aware Query Rewriting for Web Search (2026) Query Reformulation and Expansion |
|
| 27 | Jiqun Liu | 37 | 3 | 70 Toward A Two-Sided Fairness Framework in Search and Recommendation (2023) Fairness Metrics and Evaluation 41 TRUE: A Reproducible Framework for LLM-Driven Relevance Judgment in Information Retrieval (2026) LLM-Based Relevance Assessment |
|
| 28 | Gustavo Penha | 37 | 3 | 43 Contextualizing Spotify's Audiobook List Recommendations with Descriptive Shelves (2025) Recommendation Systems and Evaluation 42 Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators (2022) Query Reformulation and Expansion |
|
| 29 | Yuchen Li 0006 | 37 | 3 | 44 FULTR: A Large-Scale Fusion Learning to Rank Dataset and Its Application for Satisfaction-Oriented Ranking (2025) Web-Scale Learning to Rank 42 RankExpert: A Mixture of Textual-and-Behavioral Experts for Multi-Objective Learning-to-Rank in Web Search (2025) Unbiased Learning to Rank |
|
| 30 | Sunhao Dai | 37 | 3 | 42 Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era (2024) Bias Mitigation Methods 42 Unifying Bias and Unfairness in Information Retrieval: New Challenges in the LLM Era (2025) Bias Mitigation Methods |
|
| 31 | Sara Allawati | 37 | 3 | 42 VulGen: Workshop on Vulnerabilities in Generative Systems for Information Retrieval (2026) Bias Mitigation Methods 36 A Comparative Analysis of Linguistic and Retrieval Diversity in LLM-Generated Search Queries (2025) User Behavior and Search Interaction |