Retrieval-augmented generation (RAG) has recently emerged as a powerful framework for knowledge-intensive natural language processing tasks, which leverages the strengths of both pre-trained language models and external knowledge. While significant progress has been made, the scaling behavior of these approaches during inference remains poorly understood. Towards this end, this paper presents a comprehensive study of inference scaling law for RAG models, which investigates how inference performance scales with respect to key factors including retriever model scale, generator model scale, number of retrieved documents, and context window size. Through extensive experiments on benchmark datasets, we establish empirical scaling laws that reveal power-law and sigmoid-type relationships between these factors and performance. We further build a joint inference scaling law with theoretical justification. With the proposed scaling laws, we can understand the performance tendency of RAG models under different computational resources. We believe our insights can pave the way for efficient and effective deployment of RAG models in more applications.

Scaling laws have enabled predictable compute allocation for pre-training and for RL in reasoning tasks.However, research on retrieval reinforcement generation (RAG) remains insufficient and there is a lack of fundamental understanding of the interaction between retrieval quality and reinforcement learning computation.We present the first systematic study of RL scaling for RAG across three knowledgeintensive benchmarks.We introduce the Retrieval Bottleneck Hypothesis and derive sigmoidal scaling laws showing that retrieval quality, not RL compute, determines the asymptotic performance ceiling.Our analysis reveals three principles: (1) retrieval quality bounds achievable performance, with improving retrieval yielding larger gains than algorithmic innovations; (2) design choices (training objectives, rewards, off-policy methods) primarily modulate compute efficiency, with secondary effects on the ceiling that are substantially smaller than retrieval quality improvements; and (3) stable configurations enable extrapolation with 3.1% error at 4× compute.We further uncover RAG-specific dynamics: optimal document count increases with training, and RL algorithm effectiveness depends critically on retrieval quality.These insights yield RAG-SCALERL, achieving strong performance on knowledge-intensive benchmarks while providing the predictable scaling long available for pre-training but previously absent in RAG-RL.

Retrieval-augmented generation (RAG) has emerged as a promising solution to enhance the reliability of large language models (LLMs) with external knowledge. Existing RAG methods operate in explicit representation spaces: in-context methods inject knowledge through text tokens in the input, while parametric methods like Parametric RAG encode documents into model parameters. Although effective, these approaches face inherent limitations. In-context injection suffers from quadratic computational complexity with context length and degraded performance in complex reasoning tasks. Parametric injection, while reducing inference costs, requires substantial storage overhead and computationally expensive offline preprocessing. More fundamentally, both paradigms rely on explicit discrete representations tokens or parameters that may introduce information bottlenecks and hinder seamless knowledge integration. To address these challenges, we introduce Latent RAG, a novel paradigm that performs knowledge injection entirely within the continuous latent space. Our approach encodes documents into ultra-compact latent representations through an offline compression phase, and directly fuses them with the LLM's hidden states via a learned injection mechanism during inference. By operating in the semantic latent space rather than explicit token or parameter spaces, Latent RAG enables more natural knowledge integration while achieving 9,200X storage reduction compared to Parametric RAG. Experimental results on multiple RAG benchmarks demonstrate that Latent RAG substantially enhances both effectiveness and efficiency. Furthermore, it can be seamlessly combined with existing in-context and parametric methods to achieve even better performance.

Large Language Models (LLMs) are known to hallucinate, generating non-factual outputs that undermine user trust. Recent ensemble-based approaches leverage uncertainty estimation to select among multiple LLM responses, achieving promising results in hallucination mitigation. However, these methods treat each model's uncertainty independently, overlooking a crucial signal: cross-model consistency. In this work, we observe that answers agreed upon by multiple models are significantly more likely to be correct-a manifestation of the "wisdom of crowds" principle. Leveraging this insight, we propose Consistency-Calibrated Uncertainty Fusion (CCUF), a framework that calibrates individual model uncertainties using cross-model consistency scores. When multiple models converge on the same answer, CCUF reduces the associated uncertainty estimate; when answers diverge, uncertainty remains elevated. This calibration mechanism enables more reliable answer selection for factoid question answering. Extensive experiments on TruthfulQA, TriviaQA, and FACTOR-news benchmarks demonstrate that CCUF consistently outperforms state-of-the-art hallucination mitigation methods, surpassing the previous best ensemble method UAF by 3.4% in accuracy while exceeding GPT-4 performance on TruthfulQA by 5.2%.

Retrieval-augmented generation (RAG) has emerged as a promising paradigm to enhance Large Language Models (LLMs) with external knowledge, effectively mitigating hallucinations and broadening the model's knowledge coverage. Despite recent advances, existing RAG methods fundamentally assume static query understanding, where the query is interpreted once before retrieval. This assumption proves inadequate for multi-hop questions, where comprehending the query itself often requires retrieval support, creating a chicken-and-egg dilemma between query understanding and information retrieval. To address this challenge, we propose SCORE-RAG Self-COrrecting Exploration-Exploitation REtrieval, a novel framework inspired by the explore-exploit paradigm in decision theory. SCORE-RAG reformulates multi-hop RAG as a two-phase adaptive process: exploration for dynamic query understanding, followed by exploitation for precise evidence gathering. Specifically, SCORE-RAG first performs exploratory retrieval with multi-perspective queries to resolve ambiguities and discover key entities and relations, then conducts targeted exploitation retrieval guided by the refined understanding to construct coherent evidence chains, and finally applies self-correction mechanisms to verify consistency and repair potential errors. Through this integrated approach, SCORE-RAG enables adaptive query comprehension, reduces error accumulation via self-verification, and produces interpretable reasoning chains for accurate answer generation. Extensive experiments on HotPotQA and 2WikiMultihopQA demonstrate that SCORE-RAG significantly outperforms existing state-of-the-art RAG frameworks, achieving substantial improvements particularly on complex multi-hop questions requiring deep reasoning.

Multiparty dialogue question answering (QA) in machine reading comprehension (MRC) is a challenging task due to its complex information flow interactions and logical QA inference. Existing models typically handle such QA tasks by decoupling dialogue information at both speaker and utterance levels. However, few of them consider the logical inference relations in multiparty dialogue QA, leading to suboptimal QA performance. To address this issue, this paper proposes a memory network with logical inference (LIMN) for extractive QA in multiparty dialogues. LIMN introduces an inference module, which is pretrained by incorporating plain QA articles as external knowledge. It generates logical inference-aware representations from latent space for multiparty dialogues. To further model complex interactions among logical dialogue contexts, questions and key-utterance information, a key-utterance-based interaction method is proposed for leverage. Moreover, a multitask learning strategy is adopted for robust MRC. Extensive experiments were conducted on Molweni and FriendsQA benchmarks, which included 25k and 10k questions, respectively. Comparative results showed that LIMN achieves state-of-the-art results on both benchmarks, demonstrating the enhancement of logical QA inference in multiparty dialogue QA tasks.