Logical reasoning tasks have recently become a research hotspot in machine reading comprehension communities. This task requires models to answer the question by extracting and utilizing the implicit logical information hidden in the text. Logical information includes logical content information and logical pattern information. When the context and options are relevant in content aspect, such as Necessary Assumption, logical content information can help model to perform better. When it comes to content unrelated situation, such as Logic Principle questions, models need to further distill logical pattern information. Previous works has proposed some strategies, focusing on modeling the logical content information, but there are still some limitations such as long-distance dependency, heavy reliance on external data and internal data augmentation. In addition, the logical pattern information has not received much attention in the previous works, which will cause negative impact on the generalization in practical scenarios. In this paper, we try to improve both effectiveness and generalization of the model. We proposed a novel logical Transformer Capsule Network (LTCN). In this model, to better capture the logical content information, we combine logic graph with Transformer by using biaffine mechanism. And we fill the gap of ignoring logical pattern information by introducing a Capsule Network. Experimental results shows that our model outperforms on both ReClor and LogiQA datasets. Specially, our model has a significant performance improvements on handling more challenging logical reasoning questions.

Topic models that can take advantage of labels are broadly used in identifying interpretable topics from textual data. However, existing topic models tend to merely view labels as names of topic clusters or as categories of texts, thereby neglecting the potential causal relationships between supervised information and latent topics, as well as within these elements themselves. In this paper, we focus on uncovering possible causal relationships both between and within the supervised information and latent topics to better understand the mechanisms behind the emergence of the topics and the labels. To this end, we propose Causal Relationship-Aware Neural Topic Model (CRNTM), a novel neural topic model that can automatically uncover interpretable causal relationships between and within supervised information and latent topics, while concurrently discovering high-quality topics. In CRNTM, both supervised information and latent topics are treated as nodes, with the causal relationships represented as directed edges in a Directed Acyclic Graph (DAG). A Structural Causal Model (SCM) is employed to model the DAG. Experiments are conducted on three public corpora with different types of labels. Experimental results show that the discovered causal relationships are both reliable and interpretable, and the learned topics are of high quality comparing with eight start-of-the-art topic model baselines.

It is hard to evaluate translations objectively and accurately, which limits the applications of machine translation. In this article, we assume that the above phenomenon is caused by noise interference during translation evaluation, and we handle the problem through a perspective of causal inference. We assume that the observable translation score is affected by the unobservable true translation quality and some noise simultaneously. If there is a variable that is related to the noise and independent to the true translation quality, the related noise can be eliminated by removing the effect of that variable from the observed score. Based on the above causality hypothesis, this article studies the length bias problem of beam search for neural machine translation (NMT) and the input related noise problem of translation quality estimation (QE). For the NMT length bias problem, we conduct the experiments on four typical NMT tasks (Uyghur–Chinese, Chinese–English, English–German, and English–French) with different scales of datasets. Comparing with previous approaches, the proposed causal motivated method is model-agnostic and does not require supervised training. For QE tasks, we conduct the experiments on the WMT’20 submissions. Experimental results show that the denoised QE results gain better Pearson’s correlation scores with human assessed scores compared to the original submissions. Further analyses on the NMT and QE tasks also demonstrate the rationality of the empirical assumptions made on our methods.

Neural variational inference-based topic modeling has gained great success in mining abstract topics from documents. However, these topic models usually mainly focus on optimizing the topic proportions for documents, while the quality and the internal construction of topics are usually neglected. Specifically, these models lack the guarantee that semantically related words are supposed to be assigned to the same topic and are difficult to ensure the interpretability of topics. Moreover, many topical words recur frequently in the top words of different topics, which makes the learned topics semantically redundant and similar, and of little significance for further study. To solve the above problems, we propose a novel neural topic model called Neural Variational Gaussian Mixture Topic Model (NVGMTM). We use Gaussian distribution to depict the semantic relevance between words in the topics. Each topic in NVGMTM is considered as a multivariate Gaussian distribution over words in the word-embedding space. Thus, semantically related words share similar probabilities in each topic, which makes the topics more coherent and interpretable. Experimental results on two public corpora show the proposed model outperforms the state-of-the-art baselines.

Neural Machine Translation (NMT) optimized by Maximum Likelihood Estimation (MLE) lacks the guarantee of translation adequacy.To alleviate this problem, we propose an NMT approach that heightens the adequacy in machine translation by transferring the semantic knowledge learned from bilingual sentence alignment.Specifically, we first design a discriminator that learns to estimate sentence aligning score over translation candidates, and then the learned semantic knowledge is transfered to the NMT model under an adversarial learning framework.We also propose a gated self-attention based encoder for sentence embedding.Furthermore, an N -pair training loss is introduced in our framework to aid the discriminator in better capturing lexical evidence in translation candidates.Experimental results show that our proposed method outperforms baseline NMT models on Chinese-to-English and English-to-German translation tasks.Further analysis also indicates the detailed semantic knowledge transfered from the discriminator to the NMT model.