Large language models (LLMs) have demonstrated impressive performance in machine translation, but still struggle with unseen lowresource languages, especially those written in underrepresented scripts.To investigate whether LLMs can translate such languages with the help of linguistic resources, we introduce Lotus, a benchmark designed to evaluate translation for Mongolian (in traditional script) and Yi.Our study shows that while linguistic resources can improve translation quality as measured by automatic metrics, LLMs remain limited in their ability to handle these languages effectively.We hope our work provides insights for the low-resource NLP community and fosters further progress in machine translation for underrepresented script low-resource languages.Our code and data are available 1 .

In recent years, the Seq2Seq model has achieved satisfactory results in abstractive summarization tasks with large datasets such as Chinese or English, but this task has not been realized on low-resource Mongolian datasets. At present, the abstractive summarization methods of sota are all based on encoder-decoder architecture and they pay attention to the self-supervision goal in pre-training. Although these models can capture the context information between words in text, they still can’t integrate abstractive summarization tasks with global topic semantics. In addition, the summary generated by the autoregressive model may be inconsistent with the semantics of the source text, which leads to the low quality of the generated summarizations. To solve the above problems, we first create a dataset of Mongolian summarization, and then propose a joint learning model for abstractive summarization on this dataset. The model combines the topic model, the generation model, and the evaluation model for joint training, aiming to ensure that the generated summary not only has global topic information but also more similar to the semantics of the source text. Experiments show that this method can improve the ROUGE score, BLEU value and human evaluation score of the generated summary accordingly.

Since pre-training and fine-tuning have been a successful paradigm in Natural Language Processing (NLP), this paper adopts the SOTA pre-training model-CeMAT as a strong assistant for low-resource ethnic language translation tasks. Aiming at the exposure bias problem in the fine-tuning process, we use the contrastive learning framework and propose a new contrastive examples generation method, which uses self- generated predictions as contrastive examples to expose the model to errors during inference. Moreover, in order to effectively utilize the limited bilingual data in low-resource tasks, this paper proposes a dynamic training strategy to fine-tune the model, and refines the model step by step by taking word embedding norm and uncertainty as the criteria of evaluate data and model respectively. Experimental results demonstrate that our method significantly improves the quality compared to the baselines, which fully verifies the effectiveness.

Weak robustness and noise adaptability are major issues for Low-Resource Neural Machine Translation (NMT) models. Adversarial example is currently a major tool to improve model robustness and how to generate an adversarial examples that can degrade the performance of the model and ensure semantic consistency is a challenging task. In this paper, we adopt multi-reward reinforcement learning to generate adversarial examples for low-resource NMT. Specifically, utilizing gradient ascent to modify the source sentence, the discriminator and changes estimate are used to determine whether the generated adversarial examples maintain semantic consistency and the overall modifications of adversarial examples. Furthermore, we also install a language model reward to measure the fluency of adversarial examples. Experimental results on low-resource translation tasks show that our method highly aggressive to the model while maintaining semantic constraints greatly. Moreover, the model performance is significantly improved after fine-tuning with adversarial examples.

Multimodal Machine Translation (MMT) aims to enhance translation quality by incorporating information from other modalities (usually images). However, dominant MMT models do not consider that visual features not only provide supplementary information also introduce much noise. In this paper, we propose the visual features filter to solve this issue. Specifically, we adopt a soft-lookup function to select the visual features relevant to the text and then use these visual features as pseudo-words concatenating with a text representation. In addition, our model conducts two-pass decoding. The secondarypass decoding amounts to polishing which can identify errors in draft translations. The reason is that polishing expands the view in the process of decoding each target token, providing more contextual information. Besides, since most words in draft translations can be copied to final translations, we further equip our model with the copying mechanism to reserve those words that do not need to be corrected. MMT has achieved success in some mainstream languages at present. In order to promote the development of MMT in low-resource languages such as Mongolian, we deploy our model to the Mongolian→Chinese translation task. We expand Multi30k dataset to synthetic Mongolian and Chinese descriptions. Experiments on synthetic Mongolian and Chinese datasets demonstrate that our model can bring significant improvements.

Reinforcement Learning (RL) has been proved to alleviate metric inconsistency and exposure deviation in training-evaluation of neural machine translation (NMT), but the sample efficiency is limited by sampling methods (Temporal-Difference (TD) or Monte-Carlo (MC)), and still cannot compensate for the inefficient non-zero rewards caused by insufficient data sets. In addition, RL rewards can only be effective when the model parameters are basically determined. Therefore, we proposed episodic control reinforcement learning method, which obtains the model with basically determined parameters through the knowledge transfer, and records the historical action trajectory by introducing semi-tabular differentiable neural dictionary (DND), the model can quickly approximate the real state-value according to samples reward when updating policy. We verified on CCMT2019 Mongolian-Chinese (Mo-Zh), Tibetan-Chinese (Ti-Zh), and Uyghur-Chinese (Ug-Zh) tasks, and the results showed that the quality was significantly improved, which fully demonstrated the effectiveness of the method.