Abstract Faithfulness hallucinations , where large language models generate outputs unsupported by retrieved evidence, remain a central challenge for trustworthy AI. We present a systematic empirical evaluation of faithfulness in retrieval-augmented generation (RAG) systems using two benchmark datasets, HotpotQA and HaluBench, covering both multi-hop reasoning and single-hop hallucination detection. We analyze three small-to-mid-sized (2B-8B) open-weight LLMs in combination with multiple retrieval strategies, including sparse, dense, and hybrid approaches, as well as score-based and rank-based fusion techniques, enabling a comprehensive assessment of retrieval-generation interactions. By disentangling retrieval and generation errors, we characterize how different pipeline components contribute to hallucinations in RAG systems. Our analysis provides actionable insights and practical evaluation protocols, highlighting the critical role of robust retrieval and careful system design. These findings offer a benchmarking-oriented perspective for developing more reliable and faithful RAG systems within evaluated model scales.

In the realm of Natural Language Processing (NLP), common approaches for handling human disagreement consist of aggregating annotators' viewpoints to establish a single ground truth. However, prior studies show that disregarding individual opinions can lead to the side-effect of under-representing minority perspectives, especially in subjective tasks, where annotators may systematically disagree because of their preferences. Recognizing that labels reflect the diverse backgrounds, life experiences, and values of individuals, this study proposes a new multi-perspective approach using soft labels to encourage the development of the next generation of perspective-aware models—more inclusive and pluralistic. We conduct an extensive analysis across diverse subjective text classification tasks including hate speech, irony, abusive language, and stance detection, to highlight the importance of capturing human disagreements, often overlooked by traditional aggregation methods. Results show that the multi-perspective approach not only better approximates human label distributions, as measured by Jensen-Shannon Divergence (JSD), but also achieves superior classification performance (higher F1-scores), outperforming traditional approaches. However, our approach exhibits lower confidence in tasks like irony and stance detection, likely due to the inherent subjectivity present in the texts. Lastly, leveraging Explainable AI (XAI), we explore model uncertainty and uncover meaningful insights into model predictions. All implementation details are available at our github repo.

Crowdsourcing is a popular mechanism used for labeling tasks to produce large corpora for training. However, producing a reliable crowd labeled training corpus is challenging and resource consuming. Research on crowdsourcing has shown that label quality is much affected by worker engagement and expertise. In this study, we postulate that label quality can also be affected by inherent ambiguity of the documents to be labeled. Such ambiguities are not known in advance, of course, but, once encountered by the workers, they lead to disagreement in the labeling - a disagreement that cannot be resolved by employing more workers. To deal with this problem, we propose a crowd labeling framework: we train a disagreement predictor on a small seed of documents, and then use this predictor to decide which documents of the complete corpus should be labeled and which should be checked for document-inherent ambiguities before assigning (and potentially wasting) worker effort on them. We report on the findings of the experiments we conducted on crowdsourcing a Twitter corpus for sentiment classification.