SciCheck: Reasoning Distillation for Biomedical Claim Verification
sigir26-175 · Full research paper · Gabriel Pereira, Luciano Barbosa
Retrieval-Augmented and Open-Domain · 50% Automated Fact Checking and Claim Ve · 50%
Abstract
False claims about medical information can have damaging consequences. Although LLMs have been used to verify such claims, their success depends on access to accurate content and robust reasoning capabilities. Prior research assumes access to gold evidence during inference or relies on expensive, compute-intensive scaling strategies. However, these approaches fall short when applied to real-world verification tasks, where relevant evidence is hard to obtain and efficiency is crucial. To address this, we introduce SciCheck, a novel solution that integrates web evidence retrieval with a process of reasoning distillation. Our approach fine-tunes a small language model using reasoning traces generated by an LLM. The distillation process involves a data preparation pipeline that avoids data leakage and filters reasoning traces to retain only those leading to correct answers. It also combines web and gold evidence during training to improve robustness, while evaluation is performed with web retrieval only. We performed an extensive experimental evaluation on different claim verification datasets. The results demonstrate that SciCheck outperforms competing approaches and proprietary LLMs such as Gemini 2.5 Flash in most scenarios with lower computational cost.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Farima Fatahi Bayat | 66 | 64 FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation (2025) Automated Fact Checking and Claim Verification 64 Enhancing Language Model Factuality via Activation-Based Confidence Calibration and Guided Decoding (2024) Automated Fact Checking and Claim Verification 32 From Proof to Program: Characterizing Tool-Induced Reasoning Hallucinations in Large Language Models (2026) Reward Modeling and Process Supervision |
|
| 2 | Mariano Barone | 56 | 73 Combating Biomedical Misinformation through Multi-modal Claim Detection and Evidence-based Verification (2025) RAG for Domain Applications 71 Combining Evidence and Reasoning for Biomedical Fact-Checking (2025) Automated Fact Checking and Claim Verification 66 SOLVE-Med: Specialized Orchestration for Leading Vertical Experts Across Medical Specialties (2025) Retrieval-Augmented and Open-Domain QA |
|
| 3 | Qisheng Hu | 56 | 64 Coordinating Search-Informed Reasoning and Reasoning-Guided Search in Claim Verification (2026) Automated Fact Checking and Claim Verification 64 Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance? (2025) Automated Fact Checking and Claim Verification 47 Just What You Desire: Constrained Timeline Summarization with Self-Reflection for Enhanced Relevance (2025) Multi-hop and Complex Reasoning QA |
Area chair
| Area chair | Match | |
|---|---|---|
| Yi R. Fung 0001 | 38 |
Area-chair candidates (4)
| Yi R. Fung 0001 assigned | 38 | 10 papers | |
| Xiaoxi Li 0005 | 37 | 15 papers | |
| Zhijing Wu 0001 | 36 | 9 papers | |
| Hongjin Qian | 35 | 12 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Farima Fatahi Bayat assigned | 66 | 3 | 64 FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation (2025) Automated Fact Checking and Claim Verification 64 Enhancing Language Model Factuality via Activation-Based Confidence Calibration and Guided Decoding (2024) Automated Fact Checking and Claim Verification |
|
| 2 | Weizhi Xu 0002 | 65 | 3 | 64 Heterogeneous Graph Reasoning for Fact Checking over Texts and Tables (2024) Automated Fact Checking and Claim Verification 64 Bias Mitigation for Evidence-aware Fake News Detection by Causal Intervention (2022) Automated Fact Checking and Claim Verification |
|
| 3 | Mariano Barone assigned | 56 | 3 | 73 Combating Biomedical Misinformation through Multi-modal Claim Detection and Evidence-based Verification (2025) RAG for Domain Applications 71 Combining Evidence and Reasoning for Biomedical Fact-Checking (2025) Automated Fact Checking and Claim Verification |
|
| 4 | Qisheng Hu assigned | 56 | 3 | 64 Coordinating Search-Informed Reasoning and Reasoning-Guided Search in Claim Verification (2026) Automated Fact Checking and Claim Verification 64 Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance? (2025) Automated Fact Checking and Claim Verification |
|
| 5 | Idan Szpektor | 52 | 3 | 68 DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering (2023) Knowledge Conflict and Faithfulness 64 TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models (2023) Automated Fact Checking and Claim Verification |
|
| 6 | Chen Zhao 0013 | 51 | 3 | 64 FinDVer: Explainable Claim Verification over Long and Hybrid-content Financial Documents (2024) Automated Fact Checking and Claim Verification 64 Large Language Models Help Humans Verify Truthfulness - Except When They Are Convincingly Wrong (2024) Automated Fact Checking and Claim Verification |
|
| 7 | Farzad Shami | 49 | 3 | 64 From Single to Multi-agent Reasoning: Advancing GeneGPT for Genomics QA (2026) Retrieval-Augmented and Open-Domain QA 38 Fact Verification in Knowledge Graphs Using LLMs (2025) KG-Augmented LLM Reasoning |
|
| 8 | Jing Liu 0022 | 44 | 3 | 68 Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation (2025) Knowledge Conflict and Faithfulness 64 A Regularized Competition Model for Question Difficulty Estimation in Community Question Answering Services (2014) Retrieval-Augmented and Open-Domain QA |
|
| 9 | Bo Wang 0069 | 44 | 3 | 64 A Graph-Enhanced Defense Framework for Explainable Fake News Detection with LLM (2026) Automated Fact Checking and Claim Verification 64 Explainable Fake News Detection with Large Language Model via Defense Among Competing Wisdom (2024) Automated Fact Checking and Claim Verification |
|
| 10 | Shu Zhou 0002 | 43 | 3 | 69 Cascaded Verification Framework: A Progressive Approach for Mitigating Hallucinations in Large Language Models (2026) Knowledge Conflict and Faithfulness 69 Calibrating Uncertainty with Cross-Model Consistency for LLM Hallucination Mitigation (2026) Knowledge Conflict and Faithfulness |
|
| 11 | Wenyu Huang | 43 | 3 | 57 Prompting large language models with knowledge graphs for question answering involving long-tail facts (2025) KG-Augmented LLM Reasoning 47 Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation (2025) Multi-hop and Complex Reasoning QA |
|
| 12 | Ruichao Yang | 43 | 3 | 73 A Graph-Enhanced Defense Framework for Explainable Fake News Detection with LLM (2026) RAG for Domain Applications 64 Explainable Fake News Detection with Large Language Model via Defense Among Competing Wisdom (2024) Automated Fact Checking and Claim Verification |
|
| 13 | Armin Toroghi | 42 | 3 | 71 CoLoTa: A Dataset for Entity-based Commonsense Reasoning over Long-Tail Knowledge (2025) Automated Fact Checking and Claim Verification 71 Right for Right Reasons: Large Language Models for Verifiable Commonsense Knowledge Graph Question Answering (2024) Automated Fact Checking and Claim Verification |
|
| 14 | Venktesh V | 41 | 3 | 64 FlashCheck: Exploration of Efficient Evidence Retrieval for Fast Fact-Checking (2025) Automated Fact Checking and Claim Verification 64 The CLEF-2025 CheckThat! Lab: Subjectivity, Fact-Checking, Claim Normalization, and Retrieval (2025) Automated Fact Checking and Claim Verification |
|
| 15 | Kaixin Ma | 41 | 3 | 68 Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering (2021) Retrieval-Augmented and Open-Domain QA 64 Open Domain Question Answering with A Unified Knowledge Interface (2022) Multimodal and Graph RAG |
|
| 16 | Ori Yoran | 39 | 3 | 56 Making Retrieval-Augmented Language Models Robust to Irrelevant Context (2024) Evidence Selection and Context Refinement 52 MultiModalQA: complex question answering over text, tables and images (2021) Multimodal and Cross-Modal Retrieval › Others |
|
| 17 | Archiki Prasad | 39 | 3 | 64 MeetingQA: Extractive Question-Answering on Meeting Transcripts (2023) Retrieval-Augmented and Open-Domain QA 64 PRInTS: Reward Modeling for Long-Horizon Information Seeking (2026) Reward Modeling and Process Supervision |
|
| 18 | Xingyu Tan 0001 | 39 | 3 | 42 PRoH: Dynamic Planning and Reasoning over Knowledge Hypergraphs for Retrieval-Augmented Generation (2026) Multimodal and Graph RAG 41 HydraRAG: Structured Cross-Source Enhanced Large Language Model Reasoning (2025) KG-Augmented LLM Reasoning |
|
| 19 | Jiapu Wang | 38 | 3 | 60 Large-Small Model Synergy with Multimodal Fine-Grained Heuristics for Knowledge-Based Visual Question Answering (2025) Retrieval-Augmented and Open-Domain QA 56 Decomposing and Revising What Language Models Generate (2025) Multi-Hop and Iterative Retrieval Reasoning |
|
| 21 | Qingyi Si | 38 | 3 | 77 Are Large Language Models Table-based Fact-Checkers? (2024) Automated Fact Checking and Claim Verification 60 Combo of Thinking and Observing for Outside-Knowledge VQA (2023) Retrieval-Augmented and Open-Domain QA |
|
| 22 | Adam Jatowt | 37 | 3 | 70 ArchivalQA: A Large-scale Benchmark Dataset for Open-Domain Question Answering over Historical News Collections (2022) Multi-hop and Complex Reasoning QA 47 Inferential Question Answering (2026) Multi-hop and Complex Reasoning QA |
|
| 23 | Yu Gu 0016 | 37 | 3 | 45 Few-shot In-context Learning on Knowledge Base Question Answering (2023) Multi-hop and Complex Reasoning QA 44 ArcaneQA: Dynamic Program Induction and Contextualized Encoding for Knowledge Base Question Answering (2022) Knowledge Graph Question Answering |
|
| 24 | Jinheon Baek | 37 | 3 | 68 Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks (2023) Retrieval-Augmented and Open-Domain QA 62 Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement (2023) Conversational Search and Retrieval |
|
| 25 | Sajad Ebrahimi 0001 | 37 | 3 | 64 Peerispect: Claim Verification in Scientific Peer Reviews (2026) Automated Fact Checking and Claim Verification 33 ProActLLM: Proactive Conversational Information Seeking with Large Language Models (2025) Conversational Search and Retrieval |
|
| 26 | Falk Scholer | 37 | 3 | 70 WikiHowQA: A Comprehensive Benchmark for Multi-Document Non-Factoid Question Answering (2023) Multi-hop and Complex Reasoning QA 59 Ranking Documents by Answer-Passage Quality (2018) Retrieval-Augmented and Open-Domain QA |
|
| 28 | Qinggang Zhang | 36 | 3 | 47 Knapsack Optimization-Based Schema Linking for LLM-Based Text-to-SQL Generation (2026) Multi-hop and Complex Reasoning QA 45 Cost-efficient Knowledge-based Question Answering with Large Language Models (2024) Multi-hop and Complex Reasoning QA |
|
| 29 | Haritz Puerto | 36 | 3 | 66 MetaQA: Combining Expert Agents for Multi-Skill Question Answering (2023) Retrieval-Augmented and Open-Domain QA 33 C-SEO Bench: Does Conversational SEO Work? (2025) Reasoning Evaluation and Benchmarks |
|
| 30 | Liwei Wang 0009 | 36 | 3 | 64 MVP-Tuning: Multi-View Knowledge Retrieval with Prompt Tuning for Commonsense Reasoning (2023) Retrieval-Augmented and Open-Domain QA 47 Making Long-Context Language Models Better Multi-Hop Reasoners (2024) Multi-hop and Complex Reasoning QA |
|
| 32 | Wonduk Seo | 36 | 3 | 33 Automated Visualization Code Synthesis via Multi-path Reasoning and Feedback-Driven Optimizations (2026) Reasoning Evaluation and Benchmarks 33 A New Query Expansion Approach for Enhancing Information Retrieval via Agent-Mediated Dialogic Inquiry (2026) Query Reformulation and Expansion |
|
| 33 | Xinying Qian | 35 | 3 | 47 Beyond Timestamps: Bridging Forward and Backward Reasoning in Temporal Numerical and Relational Understanding (2026) Multi-hop and Complex Reasoning QA 41 TimeR⁴ : Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering (2024) Multi-Hop and Iterative Retrieval Reasoning |