SUMMARY: Large-scale pre-trained language models (PLMs) have advanced state-of-the-art (SOTA) performance on various biomedical text mining tasks. The power of such PLMs can be combined with the advantages of deep generative models. These are examples of these combinations. However, they are trained only on general domain text, and biomedical models are still missing. In this work, we describe BioVAE, the first large-scale pre-trained latent variable language model for the biomedical domain, which uses the OPTIMUS framework to train on large volumes of biomedical text. The model shows SOTA performance on several biomedical text mining tasks when compared to existing publicly available biomedical PLMs. In addition, our model can generate more accurate biomedical sentences than the original OPTIMUS output. AVAILABILITY AND IMPLEMENTATION: Our source code and pre-trained models are freely available: https://github.com/aistairc/BioVAE. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

MOTIVATION: Recent neural approaches on event extraction from text mainly focus on flat events in general domain, while there are less attempts to detect nested and overlapping events. These existing systems are built on given entities and they depend on external syntactic tools. RESULTS: We propose an end-to-end neural nested event extraction model named DeepEventMine that extracts multiple overlapping directed acyclic graph structures from a raw sentence. On the top of the bidirectional encoder representations from transformers model, our model detects nested entities and triggers, roles, nested events and their modifications in an end-to-end manner without any syntactic tools. Our DeepEventMine model achieves the new state-of-the-art performance on seven biomedical nested event extraction tasks. Even when gold entities are unavailable, our model can detect events from raw text with promising performance. AVAILABILITY AND IMPLEMENTATION: Our codes and models to reproduce the results are available at: https://github.com/aistairc/DeepEventMine. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Phrase-based machine translation (MT) systems require large bilingual corpora for training. Nevertheless, such large bilingual corpora are unavailable for most language pairs in the world, causing a bottleneck for the development of MT. For the Asian language pairs—Japanese, Indonesian, Malay paired with Vietnamese—they are also not excluded from the case, in which there are no large bilingual corpora on these low-resource language pairs. Furthermore, although the languages are widely used in the world, there is no prior work on MT, which causes an issue for the development of MT on these languages. In this article, we conducted an empirical study of leveraging additional resources to improve MT for the Asian low-resource language pairs: translation from Japanese, Indonesian, and Malay to Vietnamese. We propose an innovative approach that lies in two strategies of building bilingual corpora from comparable data and phrase pivot translation on existing bilingual corpora of the languages paired with English. Bilingual corpora were built from Wikipedia bilingual titles to enhance bilingual data for the low-resource languages. Additionally, we introduced a combined model of the additional resources to create an effective solution to improve MT on the Asian low-resource languages. Experimental results show the effectiveness of our systems with the improvement of +2 to +7 BLEU points. This work contributes to the development of MT on low-resource languages, especially opening a promising direction for the progress of MT on the Asian language pairs.

This paper explores a novel approach to developing a dialogue system that is able to make entertaining conversations with users. It proposes a method to improve the current goal-driven dialogue systems which supportusers for specific tasks while satisfying users’goals with entertaining conversations. It then develops a dialoguesystem in which a set of features are considered to generate entertaining conversations, while reasonably prolonging the original too short dialogue. The game refinement measure is employed for the assessment of attractiveness since the conversations in dialogue systems can be seen as the process by which a player creates shoots or moves to win a game. The dialogues generated by the proposed method are evaluated by human subjects. The results confirm the effectiveness of the proposed method. The present idea can be a promising way to realize dialogue systems with entertaining conversations although further investigations are needed.

Pivot methods have shown to be an effective solution to overcome the problem of unavailable large bilingual corpora in statistical machine translation. The representative approach of pivot methods is the phrase pivot translation which is based on common pivot phrases to produce connections between source-pivot and pivot-target phrase tables. Nevertheless, this approach produces insufficient connections behind the phrase tables because pivot phrases still contain the same meaning even when they are not matched to each other. In this work, we propose applying semantic similarity between pivot phrases to phrase pivot translation. In order to extract similar pivot phrases, we used string similarity measures for phrase similarity, and WordNet and Word2Vec were used for word similarity. The experiments show that using semantic similarity is able to extract more informative phrases, which can support for phrase pivot translation.