Shreya Havaldar, Hamidreza Alvari, John Palowitch, Mohammad Javad Hosseini, Senaka Buthpitiya, Alex Fabrikant. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Natural Language Inference (NLI) remains an important benchmark task for LLMs.NLI datasets are a springboard for transfer learning to other semantic tasks, and NLI models are standard tools for identifying the faithfulness of model-generated text.There are several large scale NLI datasets today, and models have improved greatly by hill-climbing on these collections.Yet their realistic performance on out-of-distribution/domain data is less well-understood.We explore the opportunity for synthetic high-quality datasets to adapt NLI models for zero-shot use in downstream applications across new and unseen text domains.We demonstrate a new approach for generating NLI data in diverse domains and lengths, so far not covered by existing training sets.The resulting examples have meaningful premises, the hypotheses are formed in creative ways rather than simple edits to a few premise tokens, and the labels have high accuracy.We show that models trained on this data (685K synthetic examples) have the best generalization to completely new downstream test settings.On the TRUE benchmark, a T5small model trained with our data improves around 7% on average compared to training on the best alternative dataset.The improvements are more pronounced for smaller models, while still meaningful on a T5 XXL model.We also demonstrate gains on test sets when in-domain training data is augmented with our domain-general synthetic data. * Equal contribution.1 https://gluebenchmark.com/leaderboard

Recent advances in language modeling have enabled new conversational systems.In particular, it is often desirable for people to make choices among specified options when using such systems.We address this problem of reference resolution, when people use natural expressions to choose between the entities.For example, given the choice 'Should we make a Simnel cake or a Pandan cake?' a natural response from a dialog participant may be indirect: 'let's make the green one'.Such natural expressions have been little studied for reference resolution.We argue that robustly understanding such language has large potential for improving naturalness in dialog, recommendation, and search systems.We create AltEntities 1 (Alternative Entities), a new public dataset of 42K entity pairs and expressions (referring to one entity in the pair), and develop models for the disambiguation problem.Consisting of indirect referring expressions across three domains, our corpus enables for the first time the study of how language models can be adapted to this task.We find they achieve 82%-87% accuracy in realistic settings, which while reasonable also invites further advances.

Jeremiah Milbauer, Annie Louis, Mohammad Javad Hosseini, Alex Fabrikant, Donald Metzler, Tal Schuster. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Drawing inferences between open-domain natural language predicates is a necessity for true language understanding.There has been much progress in unsupervised learning of entailment graphs for this purpose.We make three contributions: (1) we reinterpret the Distributional Inclusion Hypothesis to model entailment between predicates of different valencies, like DEFEAT(Biden, Trump) WIN(Biden);(2) we actualize this theory by learning unsupervised Multivalent Entailment Graphs of open-domain predicates; and (3) we demonstrate the capabilities of these graphs on a novel question answering task.We show that directional entailment is more helpful for inference than non-directional similarity on questions of fine-grained semantics.We also show that drawing on evidence across valencies answers more questions than by using only the same valency evidence.

Link prediction and entailment graph induction are often treated as different problems.In this paper, we show that these two problems are actually complementary.We train a link prediction model on a knowledge graph of assertions extracted from raw text.We propose an entailment score that exploits the new facts discovered by the link prediction model, and then form entailment graphs between relations.We further use the learned entailments to predict improved link prediction scores.Our results show that the two tasks can benefit from each other.The new entailment score outperforms prior state-of-the-art results on a standard entialment dataset and the new link prediction scores show improvements over the raw link prediction scores.

This paper presents a new method for learning typed entailment graphs from text. We extract predicate-argument structures from multiple-source news corpora, and compute local distributional similarity scores to learn entailments between predicates with typed arguments (e.g., person contracted disease). Previous work has used transitivity constraints to improve local decisions, but these constraints are intractable on large graphs. We instead propose a scalable method that learns globally consistent similarity scores based on new soft constraints that consider both the structures across typed entailment graphs and inside each graph. Learning takes only a few hours to run over 100K predicates and our results show large improvements over local similarity scores on two entailment data sets. We further show improvements over paraphrases and entailments from the Paraphrase Database, and prior state-of-the-art entailment graphs. We show that the entailment graphs improve performance in a downstream task.

We present a novel framework, called GRAB (GRaphical models with overlApping Blocks), to capture densely connected components in a network estimate. GRAB takes as input a data matrix of p variables and n samples, and jointly learns both a network among p variables and densely connected groups of variables (called `blocks'). GRAB has four major novelties as compared to existing network estimation methods: 1) It does not require the blocks to be given a priori. 2) Blocks can overlap. 3) It can jointly learn a network structure and overlapping blocks. 4) It solves a joint optimization problem with the block coordinate descent method that is convex in each step. We show that GRAB reveals the underlying network structure substantially better than four state-of-the-art competitors on synthetic data. When applied to cancer gene expression data, GRAB outperforms its competitors in revealing known functional gene sets and potentially novel genes that drive cancer.

This paper presents a novel approach to learning to solve simple arithmetic word problems.Our system, ARIS, analyzes each of the sentences in the problem statement to identify the relevant variables and their values.ARIS then maps this information into an equation that represents the problem, and enables its (trivial) solution as shown in Figure 1.The paper analyzes the arithmetic-word problems "genre", identifying seven categories of verbs used in such problems.ARIS learns to categorize verbs with 81.2% accuracy, and is able to solve 77.7% of the problems in a corpus of standard primary school test questions.We report the first learning results on this task without reliance on predefined templates and make our data publicly available.