Retrieval-Augmented Language Models boost task performance, owing to the retriever that provides external knowledge.Although crucial, the retriever primarily focuses on semantics relevance, which may not always be effective for generation.Thus, utility-based retrieval has emerged as a promising topic, prioritizing passages that provide valid benefits for downstream tasks.However, due to insufficient understanding, capturing passage utility accurately remains unexplored.This work proposes SCARLet, a framework for training utility-based retrievers in RALMs, which incorporates two key factors, multi-task generalization and inter-passage interaction.First, SCAR-Let constructs shared context on which training data for various tasks is synthesized.This mitigates semantic bias from context differences, allowing retrievers to focus on learning task-specific utility and generalize across tasks.Next, SCARLet uses a perturbation-based attribution method to estimate passage-level utility for shared context, which reflects interactions between passages and provides more accurate feedback.We evaluate our approach on ten datasets across various tasks, both indomain and out-of-domain, showing that retrievers trained by SCARLet consistently improve the overall performance of RALMs.

With many social problems nowadays, rumor detection in social media has become increasingly important. Previous works proposed classical and deep learning methods to extract information from features or rumor propagation structures. However, these methods either require lots of labeled data or are disturbed by noise nodes easily. To address these challenges, we propose a novel method that Disentangles graph representations with Contrastive learning for Rumor Detection (DCRD). Specifically, we design a graph contrastive learning strategy, significantly reducing the requirement of labeled data. We disentangle attention and redundant graph representations to extract intrinsic features and exclude the influence of redundant information. In addition, we utilize the disentangled two parts as hard negative samples to enhance contrastive learning further. Experiment results on two real-world datasets show that DCRD outperforms state-of-the-art models. More validation experiments demonstrate the data efficiency and robustness of our method.