We address the task of unsupervised Semantic Textual Similarity (STS) by ensembling diverse pre-trained sentence encoders into sentence meta-embeddings.We apply, extend and evaluate different meta-embedding methods from the word embedding literature at the sentence level, including dimensionality reduction (Yin and Schütze, 2016), generalized Canonical Correlation Analysis (Rastogi et al., 2015) and cross-view auto-encoders (Bollegala and Bao, 2018).Our sentence metaembeddings set a new unsupervised State of The Art (SoTA) on the STS Benchmark and on the STS12-STS16 datasets, with gains of between 3.7% and 6.4% Pearson's r over singlesource systems.

Interpretability of machine learning (ML) models becomes more relevant with their increasing adoption.In this work, we address the interpretability of ML based question answering (QA) models on a combination of knowledge bases (KB) and text documents.We adapt post hoc explanation methods such as LIME and input perturbation (IP) and compare them with the self-explanatory attention mechanism of the model.For this purpose, we propose an automatic evaluation paradigm for explanation methods in the context of QA.We also conduct a study with human annotators to evaluate whether explanations help them identify better QA models.Our results suggest that IP provides better explanations than LIME or attention, according to both automatic and human evaluation.We obtain the same ranking of methods in both experiments, which supports the validity of our automatic evaluation paradigm.

Nina Poerner, Hinrich Schütze. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Abstract In this work, we focus on the task of open-type relation argument extraction (ORAE): given a corpus, a query entity Q, and a knowledge base relation (e.g., “Q authored notable work with title X”), the model has to extract an argument of non-standard entity type (entities that cannot be extracted by a standard named entity tagger, for example, X: the title of a book or a work of art) from the corpus. We develop and compare a wide range of neural models for this task yielding large improvements over a strong baseline obtained with a neural question answering system. The impact of different sentence encoding architectures and answer extraction methods is systematically compared. An encoder based on gated recurrent units combined with a conditional random fields tagger yields the best results. We release a data set to train and evaluate ORAE, based on Wikidata and obtained by distant supervision.

The behavior of deep neural networks (DNNs) is hard to understand.This makes it necessary to explore post hoc explanation methods.We conduct the first comprehensive evaluation of explanation methods for NLP.To this end, we design two novel evaluation paradigms that cover two important classes of NLP problems: small context and large context problems.Both paradigms require no manual annotation and are therefore broadly applicable.We also introduce LIMSSE, an explanation method inspired by LIME that is designed for NLP.We show empirically that LIMSSE, LRP and DeepLIFT are the most effective explanation methods and recommend them for explaining DNNs in NLP.

Cooperative and competitive game dialogs are comparatively examined with respect to temporal, basic text-based, and dialog act characteristics.The condition-specific speaker strategies are amongst others well reflected in distinct dialog act probability distributions, which are discussed in the context of the Gricean Cooperative Principle and of Relevance Theory.Based on the extracted features, we trained Bayes classifiers and support vector machines to predict the dialog condition, that yielded accuracies from 90 to 100%.Taken together the simplicity of the condition classification task and its probabilistic expressiveness for dialog acts suggests a two-stage classification of condition and dialog acts.