In multi-hop reasoning, multi-round retrieval-augmented generation (RAG) methods typically rely on LLM-generated content as the retrieval query. However, these approaches are inherently vulnerable to knowledge overshadowing—a phenomenon where critical information is overshadowed during generation. As a result, the LLM-generated content may be incomplete or inaccurate, leading to irrelevant retrieval and causing error accumulation during the iteration process. To address this challenge, we propose ActiShade, which detects and activates overshadowed knowledge to guide large language models(LLMs) in multi-hop reasoning. Specifically, ActiShade iteratively detects the overshadowed keyphrase in the given query, retrieves documents relevant to both the query and the overshadowed keyphrase, and generates a new query based on the retrieved documents to guide the next-round iteration. By supplementing the overshadowed knowledge during the formulation of next-round queries while minimizing the introduction of irrelevant noise, ActiShade reduces the error accumulation caused by knowledge overshadowing. Extensive experiments show that ActiShade outperforms existing methods across multiple datasets and LLMs.

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by leveraging external knowledge, where retrieval accuracy directly affects generation quality. However, dense retrievers, commonly employed in RAG, suffer degraded performance in evolving corpora where new documents arrive continuously and distribution shifts accumulate over time. In such settings, continually updating retrievers is crucial, yet conventional retraining is computationally expensive and often impractical. To address this challenge, we propose FlowRAG, a lightweight and effective method for continual retriever adaptation in evolving corpora. FlowRAG augments the encoder with Layer-wise Prompt Embeddings and introduces a Cross-Layer Fusion mechanism to capture hierarchical semantic representations. In addition, a novel Generator-Guided Loss aligns retriever scores and intermediate representations with the LLM's generation likelihoods, encouraging retrieval decisions that are both semantically relevant and beneficial for generation. Experiments on datasets spanning four domains demonstrate that FlowRAG, which updates only about 0.64% of the total model parameters, consistently outperforms strong baselines in retrieval accuracy, generation quality, and robustness to forgetting in non-stationary settings.

Yuhang Tian, Dandan Song, Zhijing Wu, Pan Yang, Changzhi Zhou, Jun Yang, Hao Wang, Huipeng Ma, Chenhao Li, Luan Zhang. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Large language models (LLMs) have shown exceptional performance across various domains. However, LLMs are prone to hallucinate facts and generate non-factual responses, which can undermine their reliability in real-world applications. Current hallucination detection methods suffer from external resource demands, substantial time overhead, difficulty overcoming LLMs' intrinsic limitation, and insufficient modeling. In this paper, we propose MHAD, a novel internal-representation-based hallucination detection method. MHAD utilizes linear probing to select neurons and layers within LLMs. The selected neurons and layers are demonstrated with significant awareness of hallucinations at the initial and final generation steps. By concatenating the outputs from these selected neurons of selected layers at the initial and final generation steps, a hallucination awareness vector is formed, enabling precise hallucination detection via an MLP. Additionally, we introduce SOQHD, a novel benchmark for evaluating hallucination detection in Open-Domain QA (ODQA). Extensive experiments show that MHAD outperforms existing hallucination detection methods across multiple LLMs, demonstrating superior effectiveness.