Generative retrieval presents a promising approach to information retrieval, streamlining both indexing and retrieval processes through end-to-end optimization. This method typically involves assigning a unique identifier to each document, with the retrieval goal being the generation of the correct document identifier in response to a query. Although generative retrieval has demonstrated empirical success in various tasks, designing an effective document identifier remains a challenge. Previous studies have either depended excessively on one-to-one discrete identifiers, leading to increased retrieval latency and loss of semantics in documents or have used retrieval-agnostic dense document identifiers, which can hinder performance. To this end, we propose to integrate the benefits of generative retrieval and dense retrieval using an encoder-decoder-based pre-trained language model. Particularly, the decoder, i.e., the discrete identifier, functions as a coarse retriever, effectively reducing the retrieval space in an end-to-end manner. As a complement, the encoder, i.e., the dense vector, serves as a fine-grained retriever, efficiently and precisely ranking documents in a condensed space. Accordingly, we introduce a three-stage end-to-end learning framework that optimizes identifiers and vectors. Extensive experiments reveal that the proposed method exceeds the current models in terms of effectiveness and time efficiency, across both small and larger corpus sets.

This is an erratum for the article “Learning Discrete Identifiers and Dense Vectors for Generative Retrieval” published in ACM Trans. Inf. Syst. 44, 2, Article 42 (December 2025), 24 pages.

Although large language models (LLMs) trained on extensive multilingual corpora exhibit impressive language transfer, they often fail to respond in the user’s desired language due to corpus imbalances, an embarrassingly simple problem known as the language confusion. However, existing solutions like in-context learning and supervised fine-tuning (SFT) have drawbacks: in-context learning consumes context window space, diminishing attention as text lengthens, while SFT requires extensive, labor-intensive data collection. To overcome these limitations, we propose the language-sensitive intervention (LSI), a novel, lightweight, and label-free approach. Specifically, we analyze language confusion from a causal perspective, revealing that the training corpus’s language distribution acts as a confounder, disadvantaging languages that are underrepresented in the dataset. Then, we identify a language-sensitive dimension in the LLM’s residual stream, i.e., the language vector, which allows us to estimate the average causal effect of prompts on this dimension. During inference, we directly intervene on the language vector to generate responses in the desired language.To further advance research on this issue, we introduce a new benchmark that detects language confusion and assesses content quality. Experimental results demonstrate that our method effectively mitigates language confusion without additional complex mechanisms. Our code is available at https://github.com/SoseloX/LSI.