Multi-hop question generation focuses on generating complex questions that require reasoning over multiple pieces of information of the input passage.Current models with state-of-the-art performance have been able to generate the correct questions corresponding to the answers.However, most models can not ensure the complexity of generated questions, so they may generate shallow questions that can be answered without multi-hop reasoning.To address this challenge, we propose the CQG, which is a simple and effective controlled framework.CQG employs a simple method to generate the multi-hop questions that contain key entities in multi-hop reasoning chains, which ensure the complexity and quality of the questions.In addition, we introduce a novel controlled Transformer-based decoder to guarantee that key entities appear in the questions.Experiment results show that our model greatly improves performance, which also outperforms the state-of-the-art model about 25% by 5 BLEU points on HotpotQA 1 .

Question generation over knowledge bases (KBQG) aims at generating natural questions about a subgraph, which can be answered by a given answer entity. Existing KBQG models still face two main challenges: (1) Most models often focus on the most relevant part of the answer entity, while neglecting the rest of the subgraph. (2) There are a large number of out-of-vocabulary (OOV) predicates in real-world scenarios, which are hard to adapt for most KBQG models. To address these challenges, we propose LFKQG, a controlled generation framework for Question Generation over Knowledge Bases. (1) LFKQG employs a simple controlled generation method to generate the questions containing the critical entities in the subgraph, ensuring the question is relevant to the whole subgraph. (2) We propose an optimization strategy called local fine-tuning, which can make good use of the rich information hidden in the pre-trained model to improve the ability of the model to adapt the OOV predicates. Extensive experiments show that our method outperforms existing methods significantly on three widely-used benchmark datasets SimpleQuestion, PathQuestions, and WebQuestions.

Proof generation focuses on deductive reasoning: given a hypothesis and a set of theories, including some supporting facts and logical rules expressed in natural language, the model generates a proof tree indicating how to deduce the hypothesis from given theories.Current models with state-of-theart performance employ the stepwise method, linking an individual node to the proof step-bystep.However, these methods actually focus on generating several proof paths rather than a whole tree.To address this problem, we propose ProofInfer, which generates the proof tree via iterative hierarchical inference.At each step, ProofInfer generates the entire layer for proof tree, where all nodes in this layer are generated simultaneously.Since the conventional autoregressive generation architecture cannot simultaneously predict multiple nodes, ProofInfer employs text-to-text paradigm to avoid it.To this end, we propose a divideand-conquer algorithm to encode the proof tree as the plain text recursively without structure information loss.Experimental results show that ProofInfer significantly outperforms the state-of-the-art (SOTA) models on several widely-used datasets.In addition, ProofInfer still performs well with data-limited, achieving comparable performance to the SOTA models with only 40% of the training data. 1

Xin Zhou, Jinzhu Lu, Tao Gui, Ruotian Ma, Zichu Fei, Yuran Wang, Yong Ding, Yibo Cheung, Qi Zhang, Xuanjing Huang. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.

Natural question generation (QG) aims to generate questions from a passage, and generated questions are answered from the passage.Most models with state-of-the-art performance model the previously generated text at each decoding step.However, (1) they ignore the rich structure information that is hidden in the previously generated text.(2) they ignore the impact of copied words on the passage.We perceive that information in previously generated words serves as auxiliary information in subsequent generation.To address these problems, we design the Iterative Graph Network-based Decoder (IGND) to model the previous generation using a Graph Neural Network at each decoding step.Moreover, our graph model captures dependency relations in the passage that boost the generation.Experimental results demonstrate that our model outperforms the state-of-the-art models with sentence-level QG tasks on SQuAD and MARCO datasets.

Multi-task learning (MTL) has received considerable attention, and numerous deep learning applications benefit from MTL with multiple objectives. However, constructing multiple related tasks is difficult, and sometimes only a single task is available for training in a dataset. To tackle this problem, we explored the idea of using unsupervised clustering to construct a variety of auxiliary tasks from unlabeled data or existing labeled data. We found that some of these newly constructed tasks could exhibit semantic meanings corresponding to certain human-specific attributes, but some were non-ideal. In order to effectively reduce the impact of non-ideal auxiliary tasks on the main task, we further proposed a novel meta-learning-based multi-task learning approach, which trained the shared hidden layers on auxiliary tasks, while the meta-optimization objective was to minimize the loss on the main task, ensuring that the optimizing direction led to an improvement on the main task. Experimental results across five image datasets demonstrated that the proposed method significantly outperformed existing single task learning, semi-supervised learning, and some data augmentation methods, including an improvement of more than 9% on the Omniglot dataset.

Conditional random fields (CRF) for label decoding has become ubiquitous in sequence labeling tasks.However, the local label dependencies and inefficient Viterbi decoding have always been a problem to be solved.In this work, we introduce a novel two-stage label decoding framework to model long-term label dependencies, while being much more computationally efficient.A base model first predicts draft labels, and then a novel twostream self-attention model makes refinements on these draft predictions based on longrange label dependencies, which can achieve parallel decoding for a faster prediction.In addition, in order to mitigate the side effects of incorrect draft labels, Bayesian neural networks are used to indicate the labels with a high probability of being wrong, which can greatly assist in preventing error propagation.The experimental results on three sequence labeling benchmarks demonstrated that the proposed method not only outperformed the CRF-based methods but also greatly accelerated the inference process.* Both authors contributed equally.