Large language models (LLMs) have demonstrated impressive performance in machine translation, but still struggle with unseen lowresource languages, especially those written in underrepresented scripts.To investigate whether LLMs can translate such languages with the help of linguistic resources, we introduce Lotus, a benchmark designed to evaluate translation for Mongolian (in traditional script) and Yi.Our study shows that while linguistic resources can improve translation quality as measured by automatic metrics, LLMs remain limited in their ability to handle these languages effectively.We hope our work provides insights for the low-resource NLP community and fosters further progress in machine translation for underrepresented script low-resource languages.Our code and data are available 1 .

In high-dimensional feature extraction tasks, probabilistic methods integrated with machine learning processes have become mainstream. However, while these methods help alleviate model complexity, they often introduce additional training burdens. To simultaneously achieve effective feature extraction and reduced computational cost, we propose a novel linear dimensionality reduction method called Best Initialization Vector (BIV), which leverages the principle of basis transformation to reduce the dimensionality of images. Specifically, we exploit the properties of matrix space by initializing a vector within the space as a parameter, and applying basis transformation to perform computations. This enables extreme dimensionality reduction of high-dimensional image data into vector representations, allowing the use of NLP models for image feature extraction. Our approach significantly reduces the number of parameters while maintaining compatibility with various NLP modules. To evaluate its effectiveness, we conducted experiments on multiple datasets. The results demonstrate that our method outperforms existing mainstream approaches under extreme dimensionality reduction scenarios.

In recent years, the Seq2Seq model has achieved satisfactory results in abstractive summarization tasks with large datasets such as Chinese or English, but this task has not been realized on low-resource Mongolian datasets. At present, the abstractive summarization methods of sota are all based on encoder-decoder architecture and they pay attention to the self-supervision goal in pre-training. Although these models can capture the context information between words in text, they still can’t integrate abstractive summarization tasks with global topic semantics. In addition, the summary generated by the autoregressive model may be inconsistent with the semantics of the source text, which leads to the low quality of the generated summarizations. To solve the above problems, we first create a dataset of Mongolian summarization, and then propose a joint learning model for abstractive summarization on this dataset. The model combines the topic model, the generation model, and the evaluation model for joint training, aiming to ensure that the generated summary not only has global topic information but also more similar to the semantics of the source text. Experiments show that this method can improve the ROUGE score, BLEU value and human evaluation score of the generated summary accordingly.

Since pre-training and fine-tuning have been a successful paradigm in Natural Language Processing (NLP), this paper adopts the SOTA pre-training model-CeMAT as a strong assistant for low-resource ethnic language translation tasks. Aiming at the exposure bias problem in the fine-tuning process, we use the contrastive learning framework and propose a new contrastive examples generation method, which uses self- generated predictions as contrastive examples to expose the model to errors during inference. Moreover, in order to effectively utilize the limited bilingual data in low-resource tasks, this paper proposes a dynamic training strategy to fine-tune the model, and refines the model step by step by taking word embedding norm and uncertainty as the criteria of evaluate data and model respectively. Experimental results demonstrate that our method significantly improves the quality compared to the baselines, which fully verifies the effectiveness.

Weak robustness and noise adaptability are major issues for Low-Resource Neural Machine Translation (NMT) models. Adversarial example is currently a major tool to improve model robustness and how to generate an adversarial examples that can degrade the performance of the model and ensure semantic consistency is a challenging task. In this paper, we adopt multi-reward reinforcement learning to generate adversarial examples for low-resource NMT. Specifically, utilizing gradient ascent to modify the source sentence, the discriminator and changes estimate are used to determine whether the generated adversarial examples maintain semantic consistency and the overall modifications of adversarial examples. Furthermore, we also install a language model reward to measure the fluency of adversarial examples. Experimental results on low-resource translation tasks show that our method highly aggressive to the model while maintaining semantic constraints greatly. Moreover, the model performance is significantly improved after fine-tuning with adversarial examples.

Multimodal Machine Translation (MMT) aims to enhance translation quality by incorporating information from other modalities (usually images). However, dominant MMT models do not consider that visual features not only provide supplementary information also introduce much noise. In this paper, we propose the visual features filter to solve this issue. Specifically, we adopt a soft-lookup function to select the visual features relevant to the text and then use these visual features as pseudo-words concatenating with a text representation. In addition, our model conducts two-pass decoding. The secondarypass decoding amounts to polishing which can identify errors in draft translations. The reason is that polishing expands the view in the process of decoding each target token, providing more contextual information. Besides, since most words in draft translations can be copied to final translations, we further equip our model with the copying mechanism to reserve those words that do not need to be corrected. MMT has achieved success in some mainstream languages at present. In order to promote the development of MMT in low-resource languages such as Mongolian, we deploy our model to the Mongolian→Chinese translation task. We expand Multi30k dataset to synthetic Mongolian and Chinese descriptions. Experiments on synthetic Mongolian and Chinese datasets demonstrate that our model can bring significant improvements.

By using document-level contextual information, document-level neural machine translation can achieve better results than ordinary machine translation, but traditional document-level machine translation is difficult to focus on the contextual sentence articulation relations and deep positional relations within the discourse while utilizing document-level vocabulary, and the model can concentrate only on relatively shallow inter-sentential relations or positional information. In this paper, we consider that most adjacent sentences are connected in document translation, and such links help improve the quality of translation. We propose a document translation model that focuses more on inter-sentential relations based on the previous work, and propose two methods to strengthen the model's positional information input, and combine these two methods to enhance the traditional Transformer positional information input. This paper also proposes a method for inserting paragraph information to allow inter-sentential relations to be learned by the model, and uses the improved Transformer model for Chinese-Mongolian document translation. Experiments show that in the improved Transformer system, the BLEU scores are enhanced on the Chinese-Mongolian machine translation task after fusing positional information and inter-sentential relation information, and the translation achieves better performance.

Reinforcement Learning (RL) has been proved to alleviate metric inconsistency and exposure deviation in training-evaluation of neural machine translation (NMT), but the sample efficiency is limited by sampling methods (Temporal-Difference (TD) or Monte-Carlo (MC)), and still cannot compensate for the inefficient non-zero rewards caused by insufficient data sets. In addition, RL rewards can only be effective when the model parameters are basically determined. Therefore, we proposed episodic control reinforcement learning method, which obtains the model with basically determined parameters through the knowledge transfer, and records the historical action trajectory by introducing semi-tabular differentiable neural dictionary (DND), the model can quickly approximate the real state-value according to samples reward when updating policy. We verified on CCMT2019 Mongolian-Chinese (Mo-Zh), Tibetan-Chinese (Ti-Zh), and Uyghur-Chinese (Ug-Zh) tasks, and the results showed that the quality was significantly improved, which fully demonstrated the effectiveness of the method.

Keywords are considered to be important words in the text and can provide a concise representation of the text. With the surge of unlabeled short text on the Internet, automatic keyword extraction task has proven useful in other information processing applications. Graph-based approaches are prevalent unsupervised models for this task. However, most of these methods emphasize the importance of the relation between words without considering other importance factors. Furthermore, when measuring the importance of a word in a text, the damping factor is set to 0.85 following PageRank. To the best of our knowledge, there is no existing work investigating the impact of the damping factor on the keyword extraction task. In addition, there are few publicly available labeled Chinese short text datasets for this task. In this article, we investigate the importance parts of words in a given document and propose an improved graph-based method for keyword extraction from short documents. Moreover, we analyze the impact of importance factors on performance. We also provide annotated long and short Chinese datasets for this task. The model is performed on Chinese and English datasets, and results show that our model obtains improvements in performance over the previous unsupervised models on short documents. Comparative experiments show that the damping factor is related to the text length, which is neglected in traditional methods.

Nearly all of the work in neural machine translation (NMT) is limited to a quite restricted vocabulary, crudely treating all other words the same as an < unk > symbol. For the translation of language with abundant morphology, unknown (UNK) words also come from the misunderstanding of the translation model to the morphological changes. In this study, we explore two ways to alleviate the UNK problem in NMT: a new generative adversarial network (added value constraints and semantic enhancement) and a preprocessing technique that mixes morphological noise. The training process is like a win-win game in which the players are three adversarial sub models (generator, filter, and discriminator). In this game, the filter is to emphasize the discriminator’s attention to the negative generations that contain noise and improve the training efficiency. Finally, the discriminator cannot easily discriminate the negative samples generated by the generator with filter and human translations. The experimental results show that the proposed method significantly improves over several strong baseline models across various language pairs and the newly emerged Mongolian-Chinese task is state-of-the-art.

With the increasing numbers of user-generated content on the web, identifying the sentiment polarity of the given aspect provides more complete and in-depth results for businesses and customers. Existing deep learning methods ignore that the sentiment polarity of the target is related to the entire text structure, and prevalent approaches among them cannot effectively use the syntactic information. In this paper, we present a deep learning model that employs graph neural networks and graph-based attention mechanisms for aspect based sentiment analysis. In our work, the given text is considered as a graph based on its syntactic structure and the target is the specific region of the graph. Structural attention model and graph attention model are used to concentrate on relations between words and certain regions of the graph. We conduct comprehensive experiments on publicly accessible datasets, and results demonstrate that our model outperforms the state-of-the-art baselines.

With the amount of user-generated information on the Web, identifying the sentiment polarity of the given aspect provides more complete and in-depth results for businesses and customers. Aspect based sentiment analysis has gained increasing attention in decade years, but it remains a daunting task. Recently, approaches based on recurrent neural networks and convolutional neural networks have shown competitive results in this field. However, they don't take fully account of the entire text structure and the relation between words in a given document. In this paper, we propose a novel neural network method to address this problem, in which the text is treated as a graph and the aspect is the specific area of the graph. For the first time, we apply graph convolutional neural networks and structural attention model to aspect based sentiment analysis. Experiments on public-available datasets demonstrate the efficiency and effectiveness of our model.

Mongolian-Chinese neural machine translation (NMT) models often make mistakes in translating low-frequency words. We propose a method to alleviate this problem by improve NMT models with discrete translation lexicons that efficiently encode these low-frequency words. We describe a method to calcu-late the lexicon probability of generating the next word in the translation candi-date by using the attention vector of the NMT model to select which source word lexical probabilities the model should focus on. The method use this probabil-ity as a bias to combine with the stand-ard NMT probability. Experiments show an improvement of 4.02 BLEU score. We apply this method to large-scale corpus and improve the BLEU score. In addition, we also propose a novel approach to combine discrete probabilistic lexicons obtained from large-scale Mongolian - Chinese bilin-gual parallel corpus into NMT of small-scale corpus and enhance the perfor-mance of the system effectively.

In this paper, a translation model based on Convolutional Neural Network (CNN) architecture is introduced into the Mongolian-Chinese translation task. Mongolian language has rich morphology structure, so we use byte-pair encoding (BPE) to segment the Mongolian word. In addition, the Mongolian Correction approach is adopted to reduce coding errors occurred in Mongolian corpus. The statistics data show that BPE and Mongolian Correction are alleviate the data sparsity that results from very low-resource Mongolian-Chinese parallel corpus. On Mongolian-Chinese translation task, we achieve the best result 35.37 BLEU that exceeds the baseline system by 1.4 BLEU. In the experiments, effect of different translation granularity on the translation result is investigated. The experiment results show that sub-word unit is more suitable than word unit for Mongolian-Chinese translation.