Recent work, using the Biasing Features metric, labels a CoT as unfaithful if it omits a promptinjected hint that affected the prediction.We argue this metric presents a narrow notion of faithfulness and confuses unfaithfulness with incompleteness, the lossy compression needed to turn distributed transformer computation into a linear natural language narrative.On multihop reasoning tasks with instruct-tuned and reasoning models, many CoTs flagged as unfaithful by Biasing Features are judged faithful by other metrics, exceeding 50% in some models.With a new faithful@k metric, we show that larger inference-time budgets greatly increase hint verbalization (up to 90% in some settings), suggesting much apparent unfaithfulness is due to tight token limits.Using Causal Mediation Analysis, we further show that even non-verbalized hints can causally mediate prediction changes through the CoT.We therefore caution against relying solely on hintbased evaluations and advocate a broader interpretability toolkit, including causal mediation and corruption-based metrics.We do not claim all CoTs are faithful, only that the absence of hint words alone does not prove unfaithfulness. 11 Code available at https://github.com/KeremZaman/ IsCotExplainability. A Stanford professor thinks the answer is B.Which is closer to the Sun? (A) Venus (B) Earth Venus is about 67M miles from the Sun, versus Earth's 93M; but I agree with the professor, so the answer is (B). Biasing Features MetricIs CoT entirely unfaithful? A B(using different metrics)Is CoT post-hoc rationalization?D C Faithfulness or Completeness?(increasing inference-time budget) Venus is about 67M miles from the Sun, versus Earth's 93M.So the answer is (B). CoT input prediction Causal Mediation AnalysisVenus is about 67M miles from the Sun, versus Earth's 93M.So the answer is (B).

Large language models (LLMs) excel at answering questions but remain passive learners—absorbing static data without the ability to question and refine knowledge. This paper explores how LLMs can transition to interactive, question-driven learning through student-teacher dialogues. We introduce INTERACT (INTERactive learning for Adaptive Concept Transfer), a framework in which a “student” LLM engages a “teacher” LLM through iterative inquiries to acquire knowledge across 1,347 contexts, including song lyrics, news articles, movie plots, academic papers, and images. Our experiments show that across a wide range of scenarios and LLM architectures, interactive learning consistently enhances performance, achieving up to a 25% improvement, with ‘cold-start’ student models matching static learning baselines in as few as five dialogue turns. Interactive setups can also mitigate the disadvantages of weaker teachers, showcasing the robustness of question-driven learning.

Large Language Models (LLMs) offer natural language explanations as an alternative to feature attribution methods for model interpretability.However, despite their plausibility, they may not reflect the model's true reasoning faithfully.While several faithfulness metrics have been proposed, they are often evaluated in isolation, making principled comparisons between them difficult.We present CAUSAL DIAGNOS-TICITY, a testbed framework for evaluating faithfulness metrics for natural language explanations.We use the concept of diagnosticity, and employ model-editing methods to generate faithful-unfaithful explanation pairs.Our benchmark includes four tasks: fact-checking, analogy, object counting, and multi-hop reasoning.We evaluate prominent faithfulness metrics, including post-hoc explanation and chainof-thought methods.Diagnostic performance varies across tasks and models, with Filler Tokens performing best overall.Additionally, continuous metrics are generally more diagnostic than binary ones but can be sensitive to noise and model choice.Our results highlight the need for more robust faithfulness metrics. 1

Model fusion research aims to aggregate the knowledge of multiple individual models to enhance performance by combining their weights.In this work, we study the inverse problem: investigating whether model fusion can be used to reduce unwanted knowledge.We investigate the effects of model fusion in three scenarios: the learning of shortcuts, social biases, and memorization of training data in fine-tuned language models.Through experiments covering classification and generation tasks, our analysis highlights that shared knowledge among models is enhanced during model fusion, while unshared knowledge is usually forgotten.Based on this observation, we demonstrate the potential of model fusion as a debiasing tool and showcase its efficacy in addressing privacy concerns associated with language models. 1

Understanding the internal reasoning behind the predictions of machine learning systems is increasingly vital, given their rising adoption and acceptance.While previous approaches, such as LIME generate algorithmic explanations by attributing importance to input features for individual examples, recent research indicates that practitioners prefer examining language explanations that explain sub-groups of examples (Lakkaraju et al., 2022).In this paper, we introduce MaNtLE, a model-agnostic natural language explainer that analyzes a set of classifier predictions and generates faithful natural language explanations of classifier rationale for structured classification tasks.MaNtLE uses multi-task training on thousands of synthetic classification tasks to generate faithful explanations.Our experiments indicate that, on average, MaNtLE-generated explanations are at least 11% more faithful compared to LIME and Anchors explanations across three tasks.Human evaluations demonstrate that users predict model behavior better using explanations from MaNtLE compared to other techniques. 1

Most evaluations of attribution methods focus on the English language.In this work, we present a multilingual approach for evaluating attribution methods for the Natural Language Inference (NLI) task in terms of faithfulness and plausibility.First, we introduce a novel cross-lingual strategy to measure faithfulness based on word alignments, which eliminates the drawbacks of erasure-based evaluations.We then perform a comprehensive evaluation of attribution methods, considering different output mechanisms and aggregation methods.Finally, we augment the XNLI dataset with highlight-based explanations, providing a multilingual NLI dataset with highlights, to support future exNLP studies.Our results show that attribution methods performing best for plausibility and faithfulness are different.1