Logit Inflation in ListMLE: Theoretical Analysis and Mitigation Strategies
sigir26-152 · Full research paper · Riyaz Ahmad Bhat, Jaydeep Sen
Ranking Losses and Objectives · 100%
Abstract
Modern learning-to-rank methods often rely on listwise objectives that directly model and optimize relative document order over entire permutations. While these objectives improve ranking quality, they frequently produce models with highly inflated relevance scores whose magnitudes exceed what is necessary for meaningful document separation, leading to poor probabilistic calibration.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Zhen Qin 0001 | 48 | 100 Regression Compatible Listwise Objectives for Calibrated Ranking with Binary Relevance (2023) Ranking Losses and Objectives 100 Scale Calibration of Deep Ranking Models (2022) Ranking Losses and Objectives 100 RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses (2023) Ranking Losses and Objectives |
|
| 2 | Rolf Jagerman | 47 | 100 On Optimizing Top-K Metrics for Neural Ranking Models (2022) Ranking Losses and Objectives 75 Regression Compatible Listwise Objectives for Calibrated Ranking with Binary Relevance (2023) Ranking Losses and Objectives 68 RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses (2023) Cross-Encoder and Neural Reranking |
Area chair
| Area chair | Match | |
|---|---|---|
| Fernando Diaz 0001 | 37 |
Area-chair candidates (4)
| Fernando Diaz 0001 assigned | 37 | 12 papers | |
| Jun Xu 0001 | 32 | 2 papers | |
| Xin Xin 0003 | 31 | 10 papers | |
| Xiaoxue Zang | 30 | 4 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Sebastian Bruch 0001 | 50 | 3 | 100 A Stochastic Treatment of Learning to Rank Scoring Functions (2020) Ranking Losses and Objectives 100 An Alternative Cross Entropy Loss for Learning-to-Rank (2021) Ranking Losses and Objectives |
|
| 2 | Zhen Qin 0001 assigned | 48 | 3 | 100 Regression Compatible Listwise Objectives for Calibrated Ranking with Binary Relevance (2023) Ranking Losses and Objectives 100 Scale Calibration of Deep Ranking Models (2022) Ranking Losses and Objectives |
|
| 3 | Rolf Jagerman assigned | 47 | 3 | 100 On Optimizing Top-K Metrics for Neural Ranking Models (2022) Ranking Losses and Objectives 75 Regression Compatible Listwise Objectives for Calibrated Ranking with Binary Relevance (2023) Ranking Losses and Objectives |
|
| 4 | Yuchen Li 0006 | 43 | 3 | 51 GS2P: a generative pre-trained learning to rank model with over-parameterization for web-scale search (2024) Web-Scale Learning to Rank 51 GS2P: A Generative Pre-trained Learning to Rank Model with Over-parameterization for Web-Scale Search (Extended Abstract) (2024) Web-Scale Learning to Rank |
|
| 5 | Xuanhui Wang | 42 | 3 | 75 Regression Compatible Listwise Objectives for Calibrated Ranking with Binary Relevance (2023) Ranking Losses and Objectives 73 Learning to Rank in Theory and Practice: From Gradient Boosting to Neural Networks and Unbiased Learning (2019) Unbiased Learning to Rank |
|
| 6 | Franco Maria Nardini | 41 | 3 | 100 Selective Gradient Boosting for Effective Learning to Rank (2018) Ranking Losses and Objectives 73 Learning to Rank in Theory and Practice: From Gradient Boosting to Neural Networks and Unbiased Learning (2019) Unbiased Learning to Rank |
|
| 7 | Yimeng Bai | 41 | 3 | 61 Unconstrained Monotonic Calibration of Predictions in Deep Ranking Systems (2025) CTR and Conversion Rate Prediction 39 GradCraft: Elevating Multi-task Recommendations through Holistic Gradient Crafting (2024) Multi-Task and Multi-Objective Optimization |
|
| 8 | Zechun Niu | 41 | 3 | 42 Distributionally Robust Optimization for Unbiased Learning to Rank (2025) Unbiased Learning to Rank 41 Addressing Personalized Bias for Unbiased Learning to Rank (2025) Unbiased Learning to Rank |
|
| 9 | Andrew Parry | 38 | 3 | 100 Towards a Relevance Posterior in Neural Information Access (2026) Ranking Losses and Objectives 38 Optimal Re-Ranking Depth (2026) Query Performance Prediction |
|
| 11 | Theresia Veronika Rampisela | 36 | 3 | 36 Relevance-aware Individual Item Fairness Measures for Recommender Systems: Limitations and Usage Guidelines (2026) Fairness Metrics and Evaluation 36 Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study (2025) Fairness Metrics and Evaluation |
|
| 12 | Shuaiqiang Wang | 35 | 3 | 51 GS2P: a generative pre-trained learning to rank model with over-parameterization for web-scale search (2024) Web-Scale Learning to Rank 51 GS2P: A Generative Pre-trained Learning to Rank Model with Over-parameterization for Web-Scale Search (Extended Abstract) (2024) Web-Scale Learning to Rank |
|
| 13 | Negar Arabzadeh | 34 | 3 | 40 Learning Context-aware Term Importance for Query Performance Prediction (2026) Query Performance Prediction 40 Noisy Perturbations for Estimating Query Difficulty in Dense Retrievers (2023) Query Performance Prediction |
|
| 14 | Keping Bi | 34 | 3 | 100 Learning a Deep Listwise Context Model for Ranking Refinement (2018) Ranking Losses and Objectives 51 Extracting search-focused key n-grams for relevance ranking in web search (2012) Web-Scale Learning to Rank |
|
| 15 | Zhengyi Yang 0007 | 34 | 3 | 40 Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization (2025) Preference Alignment and Optimization 40 On Softmax Direct Preference Optimization for Recommendation (2024) Preference Alignment and Optimization |
|
| 16 | Antonio Mallia | 34 | 3 | 34 Fast and Effective Early Termination for Simple Ranking Functions (2025) Efficient Retrieval and Indexing 34 Fast Disjunctive Candidate Generation Using Live Block Filtering (2021) Efficient Retrieval and Indexing |
|
| 17 | Tetsuya Sakai | 33 | 3 | 35 Flexible Pseudo-Relevance Feedback Using Optimization Tables (2001) Query Reformulation and Expansion 35 WWW3E8: 259, 000 Relevance Labels for Studying the Effect of Document Presentation Order for Relevance Assessors (2021) Test Collection Construction and Pooling |
|
| 18 | Jizhi Zhang | 33 | 3 | 40 TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation (2023) Preference Alignment and Optimization 40 Large Language Models are Learnable Planners for Long-Term Recommendation (2024) Preference Alignment and Optimization |
|
| 19 | Corby Rosset | 33 | 3 | 100 An Axiomatic Approach to Regularizing Neural Ranking Models (2019) Ranking Losses and Objectives 34 Optimizing Query Evaluations Using Reinforcement Learning for Web Search (2018) Efficient Retrieval and Indexing |
|
| 20 | Chongming Gao | 33 | 3 | 39 SPRec: Self-Play to Debias LLM-based Recommendation (2025) Preference Alignment and Optimization 39 Process-Supervised LLM Recommenders via Flow-guided Tuning (2025) Preference Alignment and Optimization |
|
| 21 | Shengyao Zhuang | 33 | 3 | 39 R2LLMs: Retrieval and Ranking with LLMs (2025) Reasoning-Intensive Retrieval and Ranking 37 Where Relevance Emerges: A Layer-Wise Study of Internal Attention for Zero-Shot Re-Ranking (2026) LLM-Based Reranking |
|
| 22 | Jurek Leonhardt | 33 | 3 | 42 Data Augmentation for Sample Efficient and Robust Document Ranking (2024) Ranking Distillation and Adaptation 35 Extractive Explanations for Interpretable Text Ranking (2023) Cross-Encoder and Neural Reranking |
|
| 23 | Xiao Wang 0043 | 33 | 3 | 34 Neural Pseudo-Relevance Feedback Models for Sparse and Dense Retrieval (2022) Query Reformulation and Expansion 34 Effective Contrastive Weighting for Dense Query Expansion (2023) Query Reformulation and Expansion |
|
| 24 | Yuta Saito | 32 | 3 | 36 Off-Policy Evaluation of Ranking Policies under Diverse User Behavior (2023) Recommendation with Search and Ranking 34 Evaluating the Robustness of Off-Policy Evaluation (2021) Recommendation Systems and Evaluation |
|
| 26 | Xueguang Ma | 32 | 3 | 43 Tevatron: An Efficient and Flexible Toolkit for Neural Retrieval (2023) Learned Sparse Retrieval Efficiency 36 R2LLMs: Retrieval and Ranking with LLMs (2025) Reasoning-Intensive Retrieval and Ranking |
|
| 27 | Yang Zhang 0072 | 32 | 3 | 40 Reason4Rec: Deliberative User Preference Alignment of Large Language Models for Recommendation (2026) Preference Alignment and Optimization 40 TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation (2023) Preference Alignment and Optimization |
|
| 28 | Daiting Shi | 32 | 3 | 48 PRADA: Pre-Train Ranking Models With Diverse Relevance Signals Mined From Search Logs (2025) Unbiased Learning to Rank 41 Pre-trained Language Model based Ranking in Baidu Search (2021) Learned Sparse Retrieval Efficiency |
|
| 29 | Gustavo Penha | 31 | 3 | 44 On the Calibration and Uncertainty of Neural Learning to Rank Models for Conversational Search (2021) Conversational Search and Retrieval 42 Curriculum Learning Strategies for IR (2020) Ranking Distillation and Adaptation |
|
| 30 | Yongqi Li 0001 | 31 | 3 | 65 Learning to Rank in Generative Retrieval (2024) Generative Retrieval 34 Efficient Inference for Large Language Model-based Generative Recommendation (2025) Generative Retrieval and Ranking |
|
| 31 | Leif Azzopardi | 31 | 3 | 40 Query Performance Prediction: Evaluation Contrasted with Effectiveness (2010) Query Performance Prediction 40 Query Variation Performance Prediction for Systematic Reviews (2018) Query Performance Prediction |
|
| 32 | Xiao Zhang 0034 | 31 | 3 | 39 UniSAR: Modeling User Transition Behaviors between Search and Recommendation (2024) Recommendation with Search and Ranking 35 When Search Meets Recommendation: Learning Disentangled Search Representation for Recommendation (2023) Recommendation with Search and Ranking |