We study whether Large Language Models (LLMs) latently perform multi-hop reasoning with complex prompts such as "The mother of the singer of 'Superstition' is".We look for evidence of a latent reasoning pathway where an LLM (1) latently identifies "the singer of 'Superstition"' as Stevie Wonder, the bridge entity, and (2) uses its knowledge of Stevie Wonder's mother to complete the prompt.We analyze these two hops individually and consider their co-occurrence as indicative of latent multi-hop reasoning.For the first hop, we test if changing the prompt to indirectly mention the bridge entity instead of any other entity increases the LLM's internal recall of the bridge entity.For the second hop, we test if increasing this recall causes the LLM to better utilize what it knows about the bridge entity.We find strong evidence of latent multi-hop reasoning for the prompts of certain relation types, with the reasoning pathway used in more than 80% of the prompts.However, the utilization is highly contextual, varying across different types of prompts.Also, on average, the evidence for the second hop and the full multi-hop traversal is rather moderate and only substantial for the first hop.Moreover, we find a clear scaling trend with increasing model size for the first hop of reasoning but not for the second hop.Our experimental findings suggest potential challenges and opportunities for future development and applications of LLMs.

Ayyoob ImaniGooghari, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, André Martins, François Yvon, Hinrich Schütze. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Mattia Atzeni, Mikhail Plekhanov, Frederic Dreyer, Nora Kassner, Simone Merello, Louis Martin, Nicola Cancedda. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

While large language models (LLMs) are proficient at question-answering (QA), it is not always clear how (or even if) an answer follows from their latent "beliefs".This lack of interpretability is a growing impediment to widespread use of LLMs.To address this, our goals are to make model beliefs and their inferential relationships explicit, and to resolve inconsistencies that may exist, so that answers are supported by interpretable chains of reasoning drawn from a consistent network of beliefs.Our approach, which we call REFLEX, is to add a rational, self-reflecting layer on top of the LLM.First, given a question, we construct a belief graph using a backward-chaining process to materialize relevant model beliefs (including beliefs about answer candidates) and their inferential relationships.Second, we identify and minimize contradictions in that graph using a formal constraint reasoner.We find that REFLEX significantly improves consistency (by 8%-11% absolute) without harming overall answer accuracy, resulting in answers supported by faithful chains of reasoning drawn from a more consistent belief system.This suggests a new style of system architecture in which an LLM extended with a rational layer can provide an interpretable window into system beliefs, add a systematic reasoning capability, and repair latent inconsistencies present in the LLM.

Existing work on Entity Linking mostly assumes that the reference knowledge base is complete, and therefore all mentions can be linked.In practice this is hardly ever the case, as knowledge bases are incomplete and because novel concepts arise constantly.We introduce the temporally segmented Unknown Entity Discovery and Indexing (EDIN) -benchmark where unknown entities, that is entities not part of the knowledge base and without descriptions and labeled mentions, have to be integrated into an existing entity linking system.By contrasting EDIN with zero-shot entity linking, we provide insight on the additional challenges it poses.Building on denseretrieval based entity linking, we introduce the end-to-end EDIN-pipeline that detects, clusters, and indexes mentions of unknown entities in context.Experiments show that indexing a single embedding per entity unifying the information of multiple mentions works better than indexing mentions independently.

Recently, it has been found that monolingual English language models can be used as knowledge bases.Instead of structural knowledge base queries, masked sentences such as "Paris is the capital of [MASK]" are used as probes.We translate the established benchmarks TREx and GoogleRE into 53 languages.Working with mBERT, we investigate three questions.(i) Can mBERT be used as a multilingual knowledge base?Most prior work only considers English.Extending research to multiple languages is important for diversity and accessibility.(ii) Is mBERT's performance as knowledge base language-independent or does it vary from language to language?(iii) A multilingual model is trained on more text, e.g., mBERT is trained on 104 Wikipedias.Can mBERT leverage this for better performance?We find that using mBERT as a knowledge base yields varying performance across languages and pooling predictions across languages improves performance.Conversely, mBERT exhibits a language bias; e.g., when queried in Italian, it tends to predict Italy as the country of origin.

Although pretrained language models (PTLMs) contain significant amounts of world knowledge, they can still produce inconsistent answers to questions when probed, even after specialized training.As a result, it can be hard to identify what the model actually "believes" about the world, making it susceptible to inconsistent behavior and simple errors.Our goal is to reduce these problems.Our approach is to embed a PTLM in a broader system that also includes an evolving, symbolic memory of beliefs -a BeliefBank -that records but then may modify the raw PTLM answers.We describe two mechanisms to improve belief consistency in the overall system.First, a reasoning component -a weighted MaxSAT solver -revises beliefs that significantly clash with others.Second, a feedback component issues future queries to the PTLM using known beliefs as context.We show that, in a controlled experimental setting, these two mechanisms result in more consistent beliefs in the overall system, improving both the accuracy and consistency of its answers over time.This is significant as it is a first step towards PTLM-based architectures with a systematic notion of belief, enabling them to construct a more coherent picture of the world, and improve over time without model retraining.

Recent research investigates factual knowledge stored in large pretrained language models (PLMs).Instead of structural knowledge base (KB) queries, masked sentences such as "Paris is the capital of [MASK]" are used as probes.The good performance on this analysis task has been interpreted as PLMs becoming potential repositories of factual knowledge.In experiments across ten linguistically diverse languages, we study knowledge contained in static embeddings.We show that, when restricting the output space to a candidate set, simple nearest neighbor matching using static embeddings performs better than PLMs.E.g., static embeddings perform 1.6% points better than BERT while just using 0.3% of energy for training.One important factor in their good comparative performance is that static embeddings are standardly learned for a large vocabulary.In contrast, BERT exploits its more sophisticated, but expensive ability to compose meaningful representations from a much smaller subword vocabulary.

Abstract Consistency of a model—that is, the invariance of its behavior under meaning-preserving alternations in its input—is a highly desirable property in natural language processing. In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect to factual knowledge? To this end, we create ParaRel🤘, a high-quality resource of cloze-style query English paraphrases. It contains a total of 328 paraphrases for 38 relations. Using ParaRel🤘, we show that the consistency of all PLMs we experiment with is poor— though with high variance between relations. Our analysis of the representational spaces of PLMs suggests that they have a poor structure and are currently not suitable for representing knowledge robustly. Finally, we propose a method for improving model consistency and experimentally demonstrate its effectiveness.1

Abstract During production of this paper, an error was introduced to the formula on the bottom of the right column of page 1020. In the last two terms of the formula, the n and m subscripts were swapped. The correct formula is:Lc=∑n=1k∑m=n+1kDKL(Qnri∥Qmri)+DKL(Qmri∥Qnri)The paper has been updated.

Building on Petroni et al. (2019), we propose two new probing tasks analyzing factual knowledge stored in Pretrained Language Models (PLMs).(1) Negation.We find that PLMs do not distinguish between negated ("Birds cannot [MASK]") and non-negated ("Birds can [MASK]") cloze questions.(2) Mispriming.Inspired by priming methods in human psychology, we add "misprimes" to cloze questions ("Talk?Birds can [MASK]").We find that PLMs are easily distracted by misprimes.These results suggest that PLMs still have a long way to go to adequately learn human-like factual knowledge.

How can pretrained language models (PLMs) learn factual knowledge from the training set?We investigate the two most important mechanisms: reasoning and memorization.Prior work has attempted to quantify the number of facts PLMs learn, but we present, using synthetic data, the first study that investigates the causal relation between facts present in training and facts learned by the PLM.For reasoning, we show that PLMs seem to learn to apply some symbolic reasoning rules correctly but struggle with others, including two-hop reasoning.Further analysis suggests that even the application of learned reasoning rules is flawed.For memorization, we identify schema conformity (facts systematically supported by other facts) and frequency as key factors for its success.

Nowadays, classical count-based word embeddings using positive pointwise mutual information (PPMI) weighted co-occurrence matrices have been widely superseded by machine-learning-based methods like word2vec and GloVe. But these methods are usually applied using very large amounts of text data. In many cases, however, there is not much text data available, for example for specific domains or low-resource languages. This paper revisits PPMI by adding Dirichlet smoothing to correct its bias towards rare words. We evaluate on standard word similarity data sets and compare to word2vec and the recent state of the art for low-resource settings: Positive and Unlabeled (PU) Learning for word embeddings. The proposed method outperforms PU-Learning for low-resource settings and obtains competitive results for Maltese and Luxembourgish.