Reflective listening–demonstrating that you have heard your conversational partner–is key to effective communication. Expert human communicators often mimic and rephrase their conversational partner, e.g., when responding to sentimental stories or to questions they don't know the answer to. We introduce a new task and an associated dataset wherein dialogue agents similarly mimic and rephrase a user's request to communicate sympathy (I'm sorry to hear that) or lack of knowledge (I do not know that). We study what makes a rephrasal response good against a set of qualitative metrics. We then evaluate three models for generating responses: a syntax-aware rule-based system, a seq2seq LSTM neural models with attention (S2SA), and the same neural model augmented with a copy mechanism (S2SA+C). In a human evaluation, we find that S2SA+C and the rule-based system are comparable and approach human-generated response quality. In addition, experiences with a live deployment of S2SA+C in a customer support setting suggest that this generation task is a practical contribution to real world conversational agents.

Organized relational knowledge in the form of "knowledge graphs" is important for many applications.However, the ability to populate knowledge bases with facts automatically extracted from documents has improved frustratingly slowly.This paper simultaneously addresses two issues that have held back prior work.We first propose an effective new model, which combines an LSTM sequence model with a form of entity position-aware attention that is better suited to relation extraction.Then we build TACRED, a large (119,474 examples) supervised relation extraction dataset, obtained via crowdsourcing and targeted towards TAC KBP relations.The combination of better supervised data and a more appropriate high-capacity model enables much better relation extraction performance.When the model trained on this new dataset replaces the previous relation extraction component of the best TAC KBP 2015 slot filling system, its F 1 score increases markedly from 22.2% to 26.7%.

Broad domain question answering is often difficult in the absence of structured knowledge bases, and can benefit from shallow lexical methods (broad coverage) and logical reasoning (high precision).We propose an approach for incorporating both of these signals in a unified framework based on natural logic.We extend the breadth of inferences afforded by natural logic to include relational entailment (e.g., buy → own) and meronymy (e.g., a person born in a city is born the city's country).Furthermore, we train an evaluation function -akin to gameplayingto evaluate the expected truth of candidate premises on the fly.We evaluate our approach on answering multiple choice science questions, achieving strong results on the dataset.

Gabor Angeli, Melvin Jose Johnson Premkumar, Christopher D. Manning. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

Keenon Werling, Gabor Angeli, Christopher D. Manning. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

Understanding entailment and contradiction is fundamental to understanding natural language, and inference about entailment and contradiction is a valuable testing ground for the development of semantic representations.However, machine learning research in this area has been dramatically limited by the lack of large-scale resources.To address this, we introduce the Stanford Natural Language Inference corpus, a new, freely available collection of labeled sentence pairs, written by humans doing a novel grounded task based on image captioning.At 570K pairs, it is two orders of magnitude larger than all other resources of its type.This increase in scale allows lexicalized classifiers to outperform some sophisticated existing entailment models, and it allows a neural network-based model to perform competitively on natural language inference benchmarks for the first time.

Common-sense reasoning is important for AI applications, both in NLP and many vision and robotics tasks.We propose NaturalLI: a Natural Logic inference system for inferring common sense facts -for instance, that cats have tails or tomatoes are round -from a very large database of known facts.In addition to being able to provide strictly valid derivations, the system is also able to produce derivations which are only likely valid, accompanied by an associated confidence.We both show that our system is able to capture strict Natural Logic inferences on the Fra-CaS test suite, and demonstrate its ability to predict common sense facts with 49% recall and 91% precision.

Broad-coverage relation extraction either requires expensive supervised training data, or suffers from drawbacks inherent to distant supervision.We present an approach for providing partial supervision to a distantly supervised relation extractor using a small number of carefully selected examples.We compare against established active learning criteria and propose a novel criterion to sample examples which are both uncertain and representative.In this way, we combine the benefits of fine-grained supervision for difficult examples with the coverage of a large distantly supervised corpus.Our approach gives a substantial increase of 3.9% endto-end F 1 on the 2013 KBP Slot Filling evaluation, yielding a net F 1 of 37.7%.