Mitigating Bias in Large Language Model Based Question Answering through Causal Front Door Prompting
sigir26-155 · Full research paper · Yaqi Yang, Ziqi Xu, Jie Li, Chenglong Ma, Jeffrey Chan, Mark Sanderson, Xin Zheng, Yongli Ren
Multi-hop and Complex Reasoning QA · 50% Bias Mitigation Methods · 50%
Abstract
Large language models (LLMs) are widely used for question answering (QA) but can generate biased or stereotype-driven answers due to demographic associations learned during pre-training. Existing mitigation strategies often rely on model access or fine-tuning, which limits their applicability to closed-source LLMs. We propose a Causal Front Door Prompting framework (CFDP) that reduces demographic influence by intervening on the chain of thought reasoning, which is treated as an observable mediator. CFDP samples and clusters multiple reasoning traces and estimates answer probabilities through weighted aggregation. Experiments on two widely used bias-sensitive QA benchmarks, BBQ and Stereotype, across major LLMs show that CFDP consistently improves fairness metrics without sacrificing QA accuracy. Ablation and sensitivity analyses confirm the value of each component, indicating that causal intervention on reasoning provides an effective and practical approach for bias mitigation in LLM-based QA.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Adam Jatowt | 56 | 67 ArchivalQA: A Large-scale Benchmark Dataset for Open-Domain Question Answering over Historical News Collections (2022) Multi-hop and Complex Reasoning QA 65 Wrong Answers Can Also Be Useful: PlausibleQA - A Large-Scale QA Dataset with Answer Plausibility Scores (2025) Multi-hop and Complex Reasoning QA 65 Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models (2023) Multi-hop and Complex Reasoning QA |
|
| 2 | Sajad Ebrahimi 0001 | 39 | 65 Gender disentangled representation learning in neural rankers (2025) Bias Mitigation Methods 40 ProActLLM: Proactive Conversational Information Seeking with Large Language Models (2025) Conversational Search and Retrieval 38 Reinforcement Learning for Effective Few-Shot Ranking (2025) Ranking Distillation and Adaptation |
Area chair
| Area chair | Match | |
|---|---|---|
| Jun Xu 0001 | 39 |
Area-chair candidates (3)
| Xiaoxi Li 0005 | 39 | 15 papers | |
| Jun Xu 0001 assigned | 39 | 2 papers | |
| Xiaoxue Zang | 38 | 4 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Adam Jatowt assigned | 56 | 3 | 67 ArchivalQA: A Large-scale Benchmark Dataset for Open-Domain Question Answering over Historical News Collections (2022) Multi-hop and Complex Reasoning QA 65 Wrong Answers Can Also Be Useful: PlausibleQA - A Large-Scale QA Dataset with Answer Plausibility Scores (2025) Multi-hop and Complex Reasoning QA |
|
| 2 | Wenyu Huang | 53 | 3 | 65 Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation (2025) Multi-hop and Complex Reasoning QA 51 Prompting large language models with knowledge graphs for question answering involving long-tail facts (2025) KG-Augmented LLM Reasoning |
|
| 3 | Chen Zhao 0013 | 50 | 3 | 65 Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval (2021) Multi-hop and Complex Reasoning QA 65 TaPERA: Enhancing Faithfulness and Interpretability in Long-Form Table QA by Content Planning and Execution-based Reasoning (2024) Multi-hop and Complex Reasoning QA |
|
| 4 | Yu-An Liu 0028 | 48 | 3 | 61 Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-Box Neural Ranking Models (2025) Robustness and Adversarial IR 61 Robust Neural Information Retrieval: An Adversarial and Out-of-Distribution Perspective (2026) Robustness and Adversarial IR |
|
| 5 | Ori Yoran | 47 | 3 | 69 Answering Questions by Meta-Reasoning over Multiple Chains of Thought (2023) Reasoning-Intensive Retrieval and Ranking 69 Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills (2022) Multi-hop and Complex Reasoning QA |
|
| 6 | Qisheng Hu | 47 | 3 | 65 Just What You Desire: Constrained Timeline Summarization with Self-Reflection for Enhanced Relevance (2025) Multi-hop and Complex Reasoning QA 34 Coordinating Search-Informed Reasoning and Reasoning-Guided Search in Claim Verification (2026) Automated Fact Checking and Claim Verification |
|
| 7 | Sunhao Dai | 46 | 3 | 65 Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era (2024) Bias Mitigation Methods 65 Unifying Bias and Unfairness in Information Retrieval: New Challenges in the LLM Era (2025) Bias Mitigation Methods |
|
| 8 | Hao Liu 0078 | 45 | 3 | 65 Mitigating Fine-tuning Bias: A Parameter-Efficient Debiasing Framework for Large Language Models (2026) Bias Mitigation Methods 36 WeaveRec: An LLM-Based Cross-Domain Sequential Recommendation Framework with Model Merging (2026) Efficiency and Deployment |
|
| 9 | Qinggang Zhang | 45 | 3 | 65 Knapsack Optimization-Based Schema Linking for LLM-Based Text-to-SQL Generation (2026) Multi-hop and Complex Reasoning QA 62 Cost-efficient Knowledge-based Question Answering with Large Language Models (2024) Multi-hop and Complex Reasoning QA |
|
| 10 | Xi Ye 0003 | 44 | 3 | 65 Connecting Attributions and QA Model Behavior on Realistic Counterfactuals (2021) Multi-hop and Complex Reasoning QA 61 RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering (2022) Knowledge Graph Question Answering |
|
| 11 | Jialong Wu 0007 | 44 | 3 | 65 Causal Prompting: Debiasing Large Language Model Prompting Based on Front-Door Adjustment (2025) Bias Mitigation Methods 42 AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptation (2025) Query Reformulation and Expansion |
|
| 12 | Idan Szpektor | 43 | 3 | 69 MDCure: A Scalable Pipeline for Multi-Document Instruction-Following (2025) Multi-hop and Complex Reasoning QA 46 On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method (2023) Conversational Search and Retrieval |
|
| 13 | Yu Gu 0016 | 43 | 3 | 62 Few-shot In-context Learning on Knowledge Base Question Answering (2023) Multi-hop and Complex Reasoning QA 61 Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments (2023) Knowledge Graph Question Answering |
|
| 14 | Sara Allawati COI: co-author | 42 | 3 | 65 VulGen: Workshop on Vulnerabilities in Generative Systems for Information Retrieval (2026) Bias Mitigation Methods 31 A Comparative Analysis of Linguistic and Retrieval Diversity in LLM-Generated Search Queries (2025) User Behavior and Search Interaction |
|
| 15 | Xinying Qian | 42 | 3 | 65 Beyond Timestamps: Bridging Forward and Backward Reasoning in Temporal Numerical and Relational Understanding (2026) Multi-hop and Complex Reasoning QA 38 TimeR⁴ : Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering (2024) Multi-Hop and Iterative Retrieval Reasoning |
|
| 16 | Tianzhe Zhao | 41 | 3 | 69 Untargeted Adversarial Attack on Knowledge Graph Embeddings (2024) Knowledge Graph Embedding and Completion 63 Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond (2025) Reasoning Evaluation and Benchmarks |
|
| 17 | Andrew Parry | 41 | 3 | 61 Axiomatic Guidance for Efficient and Controlled Neural Search (2024) Robustness and Adversarial IR 61 Analyzing Adversarial Attacks on Sequence-to-Sequence Relevance Models (2024) Robustness and Adversarial IR |
|
| 18 | Weiwei Sun 0001 | 40 | 3 | 43 TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy (2025) Reasoning-Intensive Retrieval and Ranking 43 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Reasoning-Intensive Retrieval and Ranking |
|
| 19 | Daiting Shi | 40 | 3 | 65 A Question-Oriented Propagation Network for News Reading Comprehension (2022) Multi-hop and Complex Reasoning QA 42 Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation (2025) Evidence Selection and Context Refinement |
|
| 20 | Keqin Bao | 40 | 3 | 64 Item-side Fairness of Large Language Model-based Recommendation System (2024) Preference Alignment and Optimization 51 Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation (2023) Preference Alignment and Optimization |
|
| 21 | Falk Scholer COI: co-author | 40 | 3 | 67 WikiHowQA: A Comprehensive Benchmark for Multi-Document Non-Factoid Question Answering (2023) Multi-hop and Complex Reasoning QA 46 Ranking Documents by Answer-Passage Quality (2018) Retrieval-Augmented and Open-Domain QA |
|
| 23 | Jinheon Baek | 39 | 3 | 63 An Empirical Study of Multilingual Reasoning Distillation for Question Answering (2024) Multi-hop and Complex Reasoning QA 46 Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement (2023) Conversational Search and Retrieval |
|
| 24 | Corby Rosset | 39 | 3 | 68 Transformer-XH: Multi-Evidence Reasoning with eXtra Hop Attention (2020) Multi-Hop and Iterative Retrieval Reasoning 46 Axiomatic Preference Modeling for Longform Question Answering (2023) Retrieval-Augmented and Open-Domain QA |
|
| 26 | Sajad Ebrahimi 0001 assigned | 39 | 3 | 65 Gender disentangled representation learning in neural rankers (2025) Bias Mitigation Methods 40 ProActLLM: Proactive Conversational Information Seeking with Large Language Models (2025) Conversational Search and Retrieval |
|
| 27 | Jizhi Zhang | 39 | 3 | 64 Item-side Fairness of Large Language Model-based Recommendation System (2024) Preference Alignment and Optimization 51 Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation (2023) Preference Alignment and Optimization |
|
| 28 | Yifan Wang 0017 | 39 | 3 | 59 Improving Long-tail User CTR Prediction via Hierarchical Distribution Alignment (2025) Bias Mitigation Methods 57 Intersectional Two-sided Fairness in Recommendation (2024) Popularity Bias and Long-Tail Debiasing |
|
| 29 | Negar Arabzadeh | 38 | 3 | 44 Can QPP Choose the Right Query variant? Evaluating Query Variant Selection for RAG Pipelines (2026) Evidence Selection and Context Refinement 39 ReFormeR: Learning and Applying Explicit Query Reformulation Patterns (2026) Query Reformulation and Expansion |
|
| 30 | Yuzhong Qu | 38 | 3 | 61 Skeleton parsing for complex question answering over knowledge bases (2022) Knowledge Graph Question Answering 61 Enhancing Complex Question Answering over Knowledge Graphs through Evidence Pattern Retrieval (2024) Knowledge Graph Question Answering |
|
| 31 | Venktesh V | 38 | 3 | 56 SUNAR: Semantic Uncertainty based Neighborhood Aware Retrieval for Complex QA (2025) Multi-Hop and Iterative Retrieval Reasoning 43 The Surprising Effectiveness of Rankers trained on Expanded Queries (2024) Evidence Selection and Context Refinement |
|
| 32 | Farzad Shami | 38 | 3 | 44 From Single to Multi-agent Reasoning: Advancing GeneGPT for Genomics QA (2026) Retrieval-Augmented and Open-Domain QA 35 Doctron: A Web-based Collaborative Annotation Tool for Ground Truth Creation in IR (2025) Retrieval Toolkits and Reproducibility |