Large Language Models (LLMs) exhibit remarkable generative and reasoning capabilities, yet their outputs often reflect systematic cognitive biases analogous to those observed in human judgment. This paper investigates three interrelated forms of bias: confirmation bias, position bias, and framing bias. Through a series of controlled prompting experiments, we demonstrate that LLMs tend to reinforce the premises embedded in user queries (confirmation bias), favor initial or prominent elements within a prompt (position bias), and vary their conclusions depending on the positive or negative framing of the input (framing bias). We analyze these effects across different open LLMs: Qwen, Mistral, Gemma, Olmo, and LLama. These insights can inform better prompt engineering practices, strengthen evaluation benchmarks, and support the responsible use of LLMs in education, research, and decision-making.

As LLM-based chatbots reshape traditional information-seeking behaviours, the need for AI literacy becomes increasingly urgent. Fluent, confident AI outputs can mask errors and biases, making users' ability to question, verify, and contextualize information crucial to preventing misinformation. In this paper, we introduce IILAP+, which extends the Interactive Information Literacy Assessment Platform (IILAP) with an AI assistant to support the creation of assessment datasets for critical reading of AI-generated content. Unlike the original IILAP, which relies on teacher-curated datasets, IILAP+ uses retrieval-augmented generation grounded in tiered verification sources, decomposes answers into atomic claims verified via SPARQL queries and natural language inference, then injects controlled errors. An educator-in-the-loop interface allows instructors to configure, review, edit, and publish generated items. We describe the architecture, the error taxonomy with mappings to the ACRL Information Literacy Framework, and a comparative evaluation of two LLM backends (DeepSeek V3, GPT-4.1-nano) across two seed question sources, analysing answer quality, error injection reliability, and cost tradeoffs. Results show that AI-assisted dataset creation can reduce authoring effort while preserving key assessment affordances, provided human oversight remains essential, positioning AI as a complementary tool rather than a replacement for expert curation.

Errors in natural language generation, so-called hallucinations, remain a critical challenge, particularly in high-stakes domains such as healthcare or science communication. While several automatic metrics have been proposed to detect and quantify hallucinations, such as FactCC, QAGS, FEQA, and FactAcc, these metrics are often unavailable, difficult to reproduce, or incompatible with modern development workflows. We introduce MIRAGE, an open-source Python library designed to address these limitations. MIRAGE re-implements key hallucination evaluation metrics in a unified library built on the Hugging Face framework, offering modularity, reproducibility, and standardized inputs and outputs. By adhering to FAIR principles, MIRAGE promotes reproducibility, accelerates experimentation, and supports the development of future hallucination metrics. We validate MIRAGE by re-evaluating existing metrics on benchmark datasets, demonstrating comparable performance while significantly improving usability and transparency.

The general public often encounters complex texts but does not have the time or expertise to fully understand them, leading to the spread of misinformation. Automatic Text Simplification (ATS) helps make information more accessible, but its evaluation methods have not kept up with advances in text generation, especially with Large Language Models (LLMs). In particular, recent studies have shown that current ATS metrics do not correlate with the presence of errors. Manual inspections have further revealed a variety of errors, underscoring the need for a more nuanced evaluation framework, which is currently lacking. This resource paper addresses this gap by introducing a test collection for detecting and classifying errors in simplified texts. First, we propose a taxonomy of errors, with a formal focus on information distortion. Next, we introduce a parallel dataset of automatically simplified scientific texts. This dataset has been human-annotated with labels based on our proposed taxonomy. Finally, we analyze the quality of the dataset, and we study the performance of existing models to detect and classify errors from that taxonomy. These contributions give researchers the tools to better evaluate errors in ATS, develop more reliable models, and ultimately improve the quality of automatically simplified texts.

International audience

Despite recent advances in information retrieval and natural language processing, rhetorical devices that exploit ambiguity or subvert linguistic rules remain a challenge for such systems. However, corpus-based analysis of wordplay has been a perennial topic of scholarship in the humanities, including literary criticism, language education, and translation studies. The immense data-gathering effort required for these studies points to the need for specialized text retrieval and classification technology, and consequently for appropriate test collections. In this paper, we introduce and analyze a new dataset for research and applications in the retrieval and processing of wordplay. Developed for the JOKER track at CLEF 2023, our annotated corpus extends and improves upon past English wordplay detection datasets in several ways. First, we introduce hundreds of additional positive examples of wordplay; second, we provide French translations for the examples; and third, we provide negative examples of non-wordplay with characteristics closely matching those of the positive examples. This last feature helps ensure that AI models learn to effectively distinguish wordplay from non-wordplay, and not simply texts differing in length, style, or vocabulary. Our test collection represents then a step towards wordplay-aware multilingual information retrieval.

Sentence ordering (SO) is a key component of verbal ability. It is also crucial for automatic text generation. While numerous researchers developed various methods to automatically evaluate the informativeness of the produced contents, the evaluation of readability is usually performed manually. In contrast to that, we present a self-sufficient metric for SO assessment based on text topic-comment structure. We show that this metric has high accuracy.

This paper introduces a novel approach for document re-ranking in information retrieval based on topic-comment structure of texts. While most information retrieval models make the assumption that relevant documents are about the query and that aboutness can be captured considering bags of words only, we rather consider a more sophisticated analysis of discourse to capture document relevance by distinguishing the topic of a text from what is said about the topic (comment) in the text. The topic-comment structure of texts is extracted automatically from the first retrieved documents which are then re-ranked so that the top documents are the ones that share their topics with the query. The evaluation on TREC collections shows that the method significantly improves the retrieval performance.