FAIR-RAG: An End-to-End Framework for Mitigating Political Bias through Fair Retrieval-Augmented Generation
sigir26-127 · Full research paper · Jaebeom You, Kisung Lee, Hyuk-Yoon Kwon
Knowledge Conflict and Faithfulness · 50% Bias Mitigation Methods · 50%
Abstract
Retrieval-Augmented Generation (RAG) systems can amplify political bias from underlying web corpora. To empirically demonstrate this amplification, we first analyze 16,254 documents from the C4 dataset and 24,300 LLM-generated responses, revealing significant left-leaning and supportive stance bias that can propagate strongly from retrieval to generation. To mitigate this amplification of political bias, we propose FAIR-RAG, an end-to-end framework integrating (1) multi-LLM persona-based annotation, (2) a vector database with political-stance metadata, and (3) a multi-stage fairness engine designed for each of the three stages in RAG systems. FAIR-RAG achieves Attention Weighted Rank Fairness of 97.51 (82.1% improvement) and Perspective Balance of 51.01/82.37 (average 5.6% improvement over state-of-the-art) while maintaining high output quality (Context Precision: 0.974/0.975, Faithfulness: 0.994/0.996). Ablation studies confirm that all three components must operate collaboratively for optimal bias mitigation. This work provides a foundational framework for developing trustworthy and equitable AI information systems. All source code and experimental scripts are publicly available at: https://github.com/bigbases/FAIR-RAG.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Yu-An Liu 0028 | 57 | 67 Compete to Complete: Co-opetition Adversarial Learning for Retrieval-Augmented Generation (2026) Knowledge Conflict and Faithfulness 67 AdversarialCoT: Single-Document Retrieval Poisoning for LLM Reasoning (2026) Knowledge Conflict and Faithfulness 61 Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-Box Neural Ranking Models (2025) Robustness and Adversarial IR |
|
| 2 | Maryam Amirizaniani | 38 | 62 AuditLLM: A Tool for Auditing Large Language Models Using Multiprobe Approach (2024) Fairness Metrics and Evaluation 62 How Does Memorization Impact LLMs' Social Reasoning? An Assessment using Seen and Unseen Queries (2025) Knowledge Conflict and Faithfulness 41 Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering (2026) Personalized Question Answering and RAG |
Area chair
| Area chair | Match | |
|---|---|---|
| Xiaoxue Zang | 44 |
Area-chair candidates (7)
| Xiaoxue Zang assigned | 44 | 4 papers | |
| Xiaoxi Li 0005 | 43 | 15 papers | |
| Jun Xu 0001 | 40 | 2 papers | |
| Zhijing Wu 0001 | 40 | 9 papers | |
| Kam-Fai Wong | 38 | 6 papers | |
| Yi R. Fung 0001 | 37 | 10 papers | |
| Hongjin Qian | 37 | 12 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Yu-An Liu 0028 assigned | 57 | 3 | 67 Compete to Complete: Co-opetition Adversarial Learning for Retrieval-Augmented Generation (2026) Knowledge Conflict and Faithfulness 67 AdversarialCoT: Single-Document Retrieval Poisoning for LLM Reasoning (2026) Knowledge Conflict and Faithfulness |
|
| 2 | Sunhao Dai | 48 | 3 | 67 Unifying Bias and Unfairness in Information Retrieval: New Challenges in the LLM Era (2025) Bias Mitigation Methods 67 Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era (2024) Bias Mitigation Methods |
|
| 3 | Sara Allawati | 47 | 3 | 67 VulGen: Workshop on Vulnerabilities in Generative Systems for Information Retrieval (2026) Bias Mitigation Methods 35 A Comparative Analysis of Linguistic and Retrieval Diversity in LLM-Generated Search Queries (2025) User Behavior and Search Interaction |
|
| 4 | Di Wu 0054 | 45 | 3 | 67 Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation (2024) Knowledge Conflict and Faithfulness 40 LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (2025) Conversational Search and Retrieval |
|
| 5 | Jing Liu 0022 | 44 | 3 | 68 Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generation (2026) Evidence Selection and Context Refinement 67 Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation (2025) Knowledge Conflict and Faithfulness |
|
| 7 | Xiao Zhang 0034 | 44 | 3 | 67 ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability (2025) Knowledge Conflict and Faithfulness 67 LargePiG for Hallucination-Free Query Generation: Your Large Language Model is Secretly a Pointer Generator (2025) Knowledge Conflict and Faithfulness |
|
| 8 | Hao Liu 0078 | 44 | 3 | 67 Mitigating Fine-tuning Bias: A Parameter-Efficient Debiasing Framework for Large Language Models (2026) Bias Mitigation Methods 32 WeaveRec: An LLM-Based Cross-Domain Sequential Recommendation Framework with Model Merging (2026) Efficiency and Deployment |
|
| 9 | Jialong Wu 0007 | 43 | 3 | 67 Causal Prompting: Debiasing Large Language Model Prompting Based on Front-Door Adjustment (2025) Bias Mitigation Methods 38 PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation (2025) Efficiency and Deployment |
|
| 11 | Weiwei Sun 0001 | 43 | 3 | 67 Contrastive Learning Reduces Hallucination in Conversations (2023) Knowledge Conflict and Faithfulness 43 Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models (2026) Evidence Selection and Context Refinement |
|
| 12 | Yujia Zhou 0002 | 42 | 3 | 67 Robust Fine-tuning for Retrieval Augmented Generation against Retrieval Defects (2025) Knowledge Conflict and Faithfulness 46 Grounding Language Model with Chunking-Free In-Context Retrieval (2024) Generative Retrieval |
|
| 13 | Andrew Parry | 41 | 3 | 61 Analyzing Adversarial Attacks on Sequence-to-Sequence Relevance Models (2024) Robustness and Adversarial IR 61 Axiomatic Guidance for Efficient and Controlled Neural Search (2024) Robustness and Adversarial IR |
|
| 14 | Xi Ye 0003 | 41 | 3 | 67 Effective Large Language Model Adaptation for Improved Grounding and Citation Generation (2024) Knowledge Conflict and Faithfulness 38 Beyond Single-shot Writing: Deep Research Agents are Unreliable at Multi-turn Report Revision (2026) Deep Research and Report Generation |
|
| 15 | Yifan Wang 0017 | 41 | 3 | 60 Improving Long-tail User CTR Prediction via Hierarchical Distribution Alignment (2025) Bias Mitigation Methods 59 Intersectional Two-sided Fairness in Recommendation (2024) Popularity Bias and Long-Tail Debiasing |
|
| 16 | Keqin Bao | 40 | 3 | 62 Item-side Fairness of Large Language Model-based Recommendation System (2024) Preference Alignment and Optimization 49 Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation (2023) Preference Alignment and Optimization |
|
| 19 | Qinggang Zhang | 40 | 3 | 67 FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation (2025) Knowledge Conflict and Faithfulness 40 You Don't Need Pre-Built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures (2026) Multi-Hop and Iterative Retrieval Reasoning |
|
| 20 | Jinheon Baek | 40 | 3 | 67 Efficient Real-time Refinement of Language Model Text Generation (2025) Knowledge Conflict and Faithfulness 42 Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity (2024) Multi-Hop and Iterative Retrieval Reasoning |
|
| 21 | Shijie Wang 0002 | 40 | 3 | 62 CheatAgent: Attacking LLM-Empowered Recommender Systems via LLM Agent (2024) Preference Alignment and Optimization 50 Multi-Agent Attacks for Black-Box Social Recommendations (2025) Bias Mitigation Methods |
|
| 22 | Qingyi Si | 39 | 3 | 67 Breaking the Trade-Off Between Faithfulness and Expressiveness for Large Language Models (2026) Knowledge Conflict and Faithfulness 62 Compressing and Debiasing Vision-Language Pre-Trained Models for Visual Question Answering (2023) Multimodal Evaluation and Benchmarks |
|
| 23 | Jizhi Zhang | 39 | 3 | 62 Item-side Fairness of Large Language Model-based Recommendation System (2024) Preference Alignment and Optimization 49 Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation (2023) Preference Alignment and Optimization |
|
| 24 | Sajad Ebrahimi 0001 | 39 | 3 | 67 Gender disentangled representation learning in neural rankers (2025) Bias Mitigation Methods 38 Reinforcement Learning for Effective Few-Shot Ranking (2025) Ranking Distillation and Adaptation |
|
| 25 | Shu Zhou 0002 | 38 | 3 | 61 Calibrating Uncertainty with Cross-Model Consistency for LLM Hallucination Mitigation (2026) Knowledge Conflict and Faithfulness 61 Cascaded Verification Framework: A Progressive Approach for Mitigating Hallucinations in Large Language Models (2026) Knowledge Conflict and Faithfulness |
|
| 26 | Idan Szpektor | 38 | 3 | 67 DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering (2023) Knowledge Conflict and Faithfulness 39 MDCure: A Scalable Pipeline for Multi-Document Instruction-Following (2025) Multi-hop and Complex Reasoning QA |
|
| 28 | Kaixin Ma | 38 | 3 | 68 Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models (2024) Evidence Selection and Context Refinement 42 Dense X Retrieval: What Retrieval Granularity Should We Use? (2024) Evidence Selection and Context Refinement |
|
| 29 | Wenlin Zhang 0001 | 38 | 3 | 43 MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search (2026) Agentic and Deep Search RAG 41 To Search or Not to Search: Aligning the Decision Boundary of Deep Search Agents via Causal Intervention (2026) Agentic and Deep Search RAG |
|
| 30 | Maryam Amirizaniani assigned | 38 | 3 | 62 AuditLLM: A Tool for Auditing Large Language Models Using Multiprobe Approach (2024) Fairness Metrics and Evaluation 62 How Does Memorization Impact LLMs' Social Reasoning? An Assessment using Seen and Unseen Queries (2025) Knowledge Conflict and Faithfulness |
|
| 31 | Daiting Shi | 38 | 3 | 49 RAG-Enhanced Large Language Models for Dynamic Content Expiration Prediction in Web Search (2026) Information Retrieval and Ranking › Others 44 Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation (2025) Evidence Selection and Context Refinement |
|
| 32 | Archiki Prasad | 38 | 3 | 67 AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge (2025) Knowledge Conflict and Faithfulness 36 PRInTS: Reward Modeling for Long-Horizon Information Seeking (2026) Reward Modeling and Process Supervision |
|
| 33 | Yongqi Li 0001 | 37 | 3 | 40 Exploring Training and Inference Scaling Laws in Generative Retrieval (2025) Generative Retrieval 40 Multiview Identifiers Enhanced Generative Retrieval (2023) Generative Retrieval |
|
| 34 | Farzad Shami | 37 | 3 | 35 From Single to Multi-agent Reasoning: Advancing GeneGPT for Genomics QA (2026) Retrieval-Augmented and Open-Domain QA 34 Doctron: A Web-based Collaborative Annotation Tool for Ground Truth Creation in IR (2025) Retrieval Toolkits and Reproducibility |
|
| 35 | Haritz Puerto | 37 | 3 | 36 C-SEO Bench: Does Conversational SEO Work? (2025) Reasoning Evaluation and Benchmarks 35 Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers (2025) Fairness Metrics and Evaluation |