Non-autoregressive machine translation (NAT) has recently made great progress.However, most works to date have focused on standard translation tasks, even though some edit-based NAT models, such as the Levenshtein Transformer (LevT), seem well suited to translate with a Translation Memory (TM).This is the scenario considered here.We first analyze the vanilla LevT model and explain why it does not do well in this setting.We then propose a new variant, TM-LevT, and show how to effectively train this model.By modifying the data presentation and introducing an extra deletion operation, we obtain performance that are on par with an autoregressive approach, while reducing the decoding load.We also show that incorporating TMs during training dispenses to use knowledge distillation, a well-known trick used to mitigate the multimodality issue.

Machine Translation (MT) is usually viewed as a one-shot process that generates the target language equivalent of some source text from scratch.We consider here a more general setting which assumes an initial target sequence, that must be transformed into a valid translation of the source, thereby restoring parallelism between source and target.For this bilingual synchronization task, we consider several architectures (both autoregressive and non-autoregressive) and training regimes, and experiment with multiple practical settings such as simulated interactive MT, translating with Translation Memory (TM) and TM cleaning.Our results suggest that one single generic edit-based system, once fine-tuned, can compare with, or even outperform, dedicated systems specifically trained for these tasks.

Machine translation is generally understood as generating one target text from an input source document.In this paper, we consider a stronger requirement: to jointly generate two texts so that each output side effectively depends on the other.As we discuss, such a device serves several practical purposes, from multi-target machine translation to the generation of controlled variations of the target text.We present an analysis of possible implementations of dual decoding, and experiment with four applications.Viewing the problem from multiple angles allows us to better highlight the challenges of dual decoding and to also thoroughly analyze the benefits of generating matched, rather than independent, translations.

This paper explores data augmentation methods for training Neural Machine Translation to make use of similar translations, in a comparable way a human translator employs fuzzy matches.In particular, we show how we can simply feed the neural model with information on both source and target sides of the fuzzy matches, we also extend the similarity to include semantically related translations retrieved using distributed sentence representations.We show that translations based on fuzzy matching provide the model with "copy" information while translations based on embedding similarities tend to extend the translation "context".Results indicate that the effect from both similar sentences are adding up to further boost accuracy, are combining naturally with model fine-tuning and are providing dynamic adaptation for unseen translation pairs.Tests on multiple data sets and domains show consistent accuracy improvements.To foster research around these techniques, we also release an Open-Source toolkit with efficient and flexible fuzzy-match implementation.