Knowledge is essential for intelligent conversation systems to generate informative responses. This knowledge comprises a wide range of diverse modalities such as knowledge graphs (KGs), grounding documents and conversation topics. However, limited abilities in understanding language and utilizing different types of knowledge still challenge existing approaches. Some researchers try to enhance models’ language comprehension ability by employing the pre-trained language models, but they neglect the importance of external knowledge in specific tasks. In this paper, we propose a novel universal transformer-based architecture for dialogue system, the Multiple Knowledge Syncretic Transformer (MKST), which fuses multi-knowledge in open-domain conversation. Firstly, the model is pre-trained on a large-scale corpus to learn commonsense knowledge. Then during fine-tuning, we divide the type of knowledge into two specific categories that are handled in different ways by our model. While the encoder is responsible for encoding dialogue contexts with multifarious knowledge together, the decoder with a knowledge-aware mechanism attentively reads the fusion of multi-knowledge to promote better generation. This is the first attempt that fuses multi-knowledge in one conversation model. The experimental results have been demonstrated that our model achieves significant improvement on knowledge-driven dialogue generation tasks than state-of-the-art baselines. Meanwhile, our new benchmark could facilitate the further study in this research area.

Jinxin Chang, Ruifang He, Longbiao Wang, Xiangyu Zhao, Ting Yang, Ruifang Wang. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Open-domain multi-turn dialogue systems are booming in human-machine interactions, which encourage to chat actively and freely in an intelligent natural way. Previous generative conversational models usually employ a single and deterministic encoder-decoder framework to model the semantic consistency between the context and corresponding response. However, they neglect the various dialog patterns (we denote the regularity of topic shifting as dialog pattern) in the conversations, leading to uninformative, non-initiative yet plausible responses. Although the existing variational methods have improved the response diversity to some extent by introducing a global variability into the generative process, they fail to simulate the transfer between topics with directional information due to the weak interpretability of the Gaussian-distributed latent variables. In this paper, we propose a novel Neural Variational Scaling Reasoning Network (NVSRN) for initiative response generation. To this end, our approach has two core ingredients: neural dialog pattern reasoner (reasoner) and topic scaling mechanism. Specifically, inspired by the advantage of von Mises-Fisher (vMF) distribution modeling the directional data (e.g., the topic transfer state), we employ it as the latent space of the reasoner to explore the regularity of topic shifting, which is then used to reason the topic of response. Based on this, a topic scaling mechanism is designed to control the transfer degree of topic in the response generator. The experimental results on two large dialog datasets demonstrate that the proposed model outperforms state-of-the-art baselines. The human evaluation shows the proposed model can produce more informative and initiative responses actively.