Shaomu Tan, Dawei Zhu, Ke Tran, Michael Denkowski, Sony Trenous, Leonardo F. R. Ribeiro, Bill Byrne, Felix Hieber. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Tobias Domhan, Eva Hasler, Ke Tran, Sony Trenous, Bill Byrne, Felix Hieber. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Building neural machine translation systems to perform well on a specific target domain is a well-studied problem.Optimizing system performance for multiple, diverse target domains however remains a challenge.We study this problem in an adaptation setting where the goal is to preserve the existing system quality while incorporating data for domains that were not the focus of the original translation system.We find that we can improve over the performance trade-off offered by Elastic Weight Consolidation with a relatively simple data mixing strategy.At comparable performance on the new domains, catastrophic forgetting is mitigated significantly on strong WMT baselines.Combining both approaches improves the Pareto frontier on this task.

We present Sockeye 2, a modernized and streamlined version of the Sockeye neural machine translation (NMT) toolkit. New features include a simplified code base through the use of MXNet’s Gluon API, a focus on state of the art model architectures, and distributed mixed precision training. These improvements result in faster training and inference, higher automatic metric scores, and a shorter path from research to production.

The performance of Neural Machine Translation (NMT) models relies heavily on the availability of sufficient amounts of parallel data, and an efficient and effective way of leveraging the vastly available amounts of monolingual data has yet to be found.We propose to modify the decoder in a neural sequence-to-sequence model to enable multi-task learning for two strongly related tasks: target-side language modeling and translation.The decoder predicts the next target word through two channels, a target-side language model on the lowest layer, and an attentional recurrent model which is conditioned on the source representation.This architecture allows joint training on both large amounts of monolingual and moderate amounts of bilingual data to improve NMT performance.Initial results in the news domain for three language pairs show moderate but consistent improvements over a baseline trained on bilingual data only.

Current approaches to cross-lingual information retrieval (CLIR) rely on standard retrieval models into which query translations by statistical machine translation (SMT) are integrated at varying degree.In this paper, we present an attempt to turn this situation on its head: Instead of the retrieval aspect, we emphasize the translation component in CLIR.We perform search by using an SMT decoder in forced decoding mode to produce a bag-ofwords representation of the target documents to be ranked.The SMT model is extended by retrieval-specific features that are optimized jointly with standard translation features for a ranking objective.We find significant gains over the state-of-the-art in a large-scale evaluation on cross-lingual search in the domains patents and Wikipedia.

The statistical machine translation (SMT) component of cross-lingual information retrieval (CLIR) systems is often regarded as black box that is optimized for translation quality independent from the retrieval task. In recent work [10], SMT has been tuned for retrieval by training a reranker on $k$-best translations ordered according to their retrieval performance. In this paper we propose a decomposable proxy for retrieval quality that obviates the need for costly intermediate retrieval. Furthermore, we explore the full search space of the SMT decoder by directly optimizing decoder parameters under a retrieval-based objective. Experimental results for patent retrieval show our approach to be a promising alternative to the standard pipeline approach.

We present an approach to learning bilingual n-gram correspondences from relevance rankings of English documents for Japanese queries.We show that directly optimizing cross-lingual rankings rivals and complements machine translation-based cross-language information retrieval (CLIR).We propose an efficient boosting algorithm that deals with very large cross-product spaces of word correspondences.We show in an experimental evaluation on patent prior art search that our approach, and in particular a consensus-based combination of boosting and translation-based approaches, yields substantial improvements in CLIR performance.Our training and test data are made publicly available.