Large language models (LLMs) exhibit strong general intelligence, yet their multilingual performance remains highly imbalanced.Although LLMs encode substantial cross-lingual knowledge in a unified semantic space, they often struggle to reliably interface this knowledge with low-resource or unseen languages.Fortunately, pretrained encoder-decoder translation models already possess balanced multilingual capability, suggesting a natural complement to LLMs.In this work, we propose XBridge, a compositional encoder-LLM-decoder architecture that offloads multilingual understanding and generation to external pretrained translation models, while preserving the LLM as an English-centric core for general knowledge processing.To address the resulting representation misalignment across models, we introduce lightweight cross-model mapping layers and an optimal transport-based alignment objective, enabling fine-grained semantic consistency for multilingual generation.Experiments on four LLMs across multilingual understanding, reasoning, summarization, and generation indicate that XBridge outperforms strong baselines, especially on low-resource and previously unseen languages, without retraining the LLM.

Multilingual large language models (LLMs) possess impressive multilingual understanding and generation capabilities.However, their performance and cross-lingual alignment often lag for non-dominant languages.A common solution is to fine-tune LLMs on largescale and more balanced multilingual corpora, but such approaches often lead to imprecise alignment and suboptimal knowledge transfer, struggling with limited improvements across languages.In this paper, we propose AlignX to bridge the multilingual performance gap, which is a two-stage representationlevel framework for enhancing multilingual performance of pre-trained LLMs.In the first stage, we align multilingual representations with multilingual semantic alignment and language feature integration.In the second stage, we stimulate the multilingual capability of LLMs via multilingual instruction fine-tuning.Experimental results on several pre-trained LLMs demonstrate that our approach enhances LLMs' multilingual general and cross-lingual generation capability.Further analysis indicates that AlignX brings the multilingual representations closer and improves the cross-lingual alignment.1

Langlin Huang, Mengyu Bu, Yang Feng. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.