A central question in multilingual language modeling is whether large language models (LLMs) develop a universal concept representation, disentangled from specific languages.In this paper, we address this question by analyzing latent representations (latents) during a word-translation task in transformer-based LLMs.We strategically extract latents from a source translation prompt and insert them into the forward pass on a target translation prompt.By doing so, we find that the output language is encoded in the latent at an earlier layer than the concept to be translated.Building on this insight, we conduct two key experiments.First, we demonstrate that we can change the concept without changing the language and vice versa through activation patching alone.Second, we show that patching with the mean representation of a concept across different languages does not affect the models' ability to translate it, but instead improves it.Finally, we generalize to multi-token generation and demonstrate that the model can generate natural language description of those mean representations.Our results provide evidence for the existence of language-agnostic concept representations within the investigated models.

When making predictions, a language model must trade off how much it relies on its context vs. its prior knowledge. Choosing how sensitive the model is to its context is a fundamental functionality, as it enables the model to excel at tasks like retrieval-augmented generation and question-answering. In this paper, we search for a knob which controls this sensitivity, determining whether language models answer from the context or their prior knowledge. To guide this search, we design a task for controllable context sensitivity. In this task, we first feed the model a context ("Paris is in England") and a question ("Where is Paris?"); we then instruct the model to either use its prior or contextual knowledge and evaluate whether it generates the correct answer for both intents (either "France" or "England"). When fine-tuned on this task, instruct versions of Llama-3.1, Mistral-v0.3, and Gemma-2 can solve it with high accuracy (85-95%). Analyzing these high-performing models, we narrow down which layers may be important to context sensitivity using a novel linear time algorithm. Then, in each model, we identify a 1-D subspace in a single layer that encodes whether the model follows context or prior knowledge. Interestingly, while we identify this subspace in a fine-tuned model, we find that the exact same subspace serves as an effective knob in not only that model but also non-fine-tuned instruct and base models of that model family. Finally, we show a strong correlation between a model's performance and how distinctly it separates context-agreeing from context-ignoring answers in this subspace. These results suggest a single fundamental subspace facilitates how the model chooses between context and prior knowledge.

Giovanni Monea, Maxime Peyrard, Martin Josifoski, Vishrav Chaudhary, Jason Eisner, Emre Kiciman, Hamid Palangi, Barun Patra, Robert West. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

We ask whether multilingual language models trained on unbalanced, English-dominated corpora use English as an internal pivot languagea question of key importance for understanding how language models function and the origins of linguistic bias.Focusing on the Llama-2 family of transformer models, our study uses carefully constructed non-English prompts with a unique correct single-token continuation.From layer to layer, transformers gradually map an input embedding of the final prompt token to an output embedding from which next-token probabilities are computed.Tracking intermediate embeddings through their high-dimensional space reveals three distinct phases, whereby intermediate embeddings (1) start far away from output token embeddings; (2) already allow for decoding a semantically correct next token in middle layers, but give higher probability to its version in English than in the input language;(3) finally move into an input-language-specific region of the embedding space.We cast these results into a conceptual model where the three phases operate in "input space", "concept space", and "output space", respectively.Crucially, our evidence suggests that the abstract "concept space" lies closer to English than to other languages, which may have important consequences regarding the biases held by multilingual language models.Code and data is made available here: https://github.com/ epfl-dlab/llm-latent-language.0 5 10 15 20 25 30 layer 0.0 0.5 1.0 probability en zh (a) Translation task 0 5 10 15 20 25 30 35 40 layer 0.0 0.5 1.0 en zh 0 10 20 30 40 50 60 70 80 layer 0.0 0.5 1.0 en zh 0 5 10 entropy 0 5 10 15 20 25 30 layer 0.0 0.5 1.0 probability en zh (b) Repetition task 0 5 10 15 20 25 30 35 40 layer 0.0 0.5 1.0 en zh 0 10 20 30 40 50 60 70 80 layer 0.0 0.5 1.0 en zh 0 5 10 entropy 0 5 10 15 20 25 30 layer 0.0 0.5 1.0 probability en zh 7B (c) Cloze task 0 5 10 15 20 25 30 35 40 layer 0.0 0.5 1.0 en zh 13B 0 10 20 30 40 50 60 70 80 layer 0.0 0.5