Matthias Orlikowski, Jiaxin Pei, Paul Röttger, Philipp Cimiano, David Jurgens, Dirk Hovy. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Peer Reviewed

Peer Reviewed

Yinuo Xu, Hong Chen, Sushrita Rakshit, Aparna Ananthasubramaniam, Omkar Yadav, Mingqian Zheng, Michael Jiang, Lechen Zhang, Bowen Yi, Kenan Alkiek, Abraham Israeli, Bangzhao Shu, Hua Shen, Jiaxin Pei, Haotian Zhang, Miriam Schirmer, David Jurgens. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Long-horizon tasks requiring multi-step reasoning and dynamic re-planning remain challenging for large language models (LLMs). Sequential prompting methods are prone to context drift, loss of goal information, and recurrent failure cycles, while hierarchical prompting methods often weaken cross-level continuity or incur substantial runtime overhead. We introduce ReCAP (Recursive Context-Aware Reasoning and Planning), a hierarchical framework with shared context for reasoning and planning in LLMs. ReCAP combines three key mechanisms: (i) plan-ahead decomposition, in which the model generates a full subtask list, executes the first item, and refines the remainder; (ii) structured re-injection of parent plans, maintaining consistent multi-level context during recursive return; and (iii) memory-efficient execution, bounding the active prompt so costs scale linearly with task depth. Together these mechanisms align high-level goals with low-level actions, reduce redundant prompting, and preserve coherent context updates across recursion. Experiments demonstrate that ReCAP substantially improves subgoal alignment and success rates on various long-horizon reasoning benchmarks, achieving a 32\% gain on synchronous Robotouille and a 29\% improvement on asynchronous Robotouille under the strict pass@1 protocol.

Large language models (LLMs) have been shown to perform well at a variety of syntactic, discourse, and reasoning tasks.While LLMs are increasingly deployed in many forms including conversational agents that interact with humans, we lack a grounded benchmark to measure how well LLMs understand social language.Here, we introduce a new theorydriven benchmark, SOCKET, that contains 58 NLP tasks testing social knowledge which we group into five categories: humor & sarcasm, offensiveness, sentiment & emotion, trustworthiness, and other social factors.In tests on the benchmark, we demonstrate that current models attain only moderate performance but reveal significant potential for task transfer among different types and categories of tasks, which were predicted from theory.Through zero-shot evaluations, we show that pretrained models already possess some innate but limited capabilities of social language understanding and training on one category of tasks can improve zero-shot testing on others.Our benchmark provides a systematic way to analyze model performance on an important dimension of language and points to clear room for improvement to build more socially-aware LLMs.The resources are released at https://github. com/minjechoi/SOCKET.

Whether the media faithfully communicate scientific information has long been a core issue to the science community.Automatically identifying paraphrased scientific findings could enable large-scale tracking and analysis of information changes in the science communication process, but this requires systems to understand the similarity between scientific information across multiple domains.To this end, we present the SCIENTIFIC PARAPHRASE AND INFOR-MATION CHANGE DATASET (SPICED), the first paraphrase dataset of scientific findings annotated for degree of information change.SPICED contains 6,000 scientific finding pairs extracted from news stories, social media discussions, and full texts of original papers.We demonstrate that SPICED poses a challenging task and that models trained on SPICED improve downstream performance on evidence retrieval for fact checking of real-world scientific claims.Finally, we show that models trained on SPICED can reveal large-scale trends in the degrees to which people and organizations faithfully communicate new scientific findings.Data, code, and pre-trained models are available at http://www.copenlu.com/publication/ 2022_emnlp_wright/.

Certainty and uncertainty are fundamental to science communication.Hedges have widely been used as proxies for uncertainty.However, certainty is a complex construct, with authors expressing not only the degree but the type and aspects of uncertainty in order to give the reader a certain impression of what is known.Here, we introduce a new study of certainty that models both the level and the aspects of certainty in scientific findings.Using a new dataset of 2167 annotated scientific findings, we demonstrate that hedges alone account for only a partial explanation of certainty.We show that both the overall certainty and individual aspects can be predicted with pre-trained language models, providing a more complete picture of the author's intended communication.Downstream analyses on 431K scientific findings from news and scientific abstracts demonstrate that modeling sentencelevel and aspect-level certainty is meaningful for areas like science communication.Both the model and datasets used in this paper are released at https://blablablab.si. umich.edu/projects/certainty/

Conditional Text Generation has drawn much attention as a topic of Natural Language Generation (NLG) which provides the possibility for humans to control the properties of generated contents.Current conditional generation models cannot handle emerging conditions due to their joint end-to-end learning fashion.When a new condition added, these techniques require full retraining.In this paper, we present a new framework named Pre-train and Plug-in Variational Auto-Encoder (PPVAE) towards flexible conditional text generation.PPVAE decouples the text generation module from the condition representation module to allow "one-to-many" conditional generation.When a fresh condition emerges, only a lightweight network needs to be trained and works as a plug-in for PPVAE, which is efficient and desirable for real-world applications.Extensive experiments demonstrate the superiority of PPVAE against the existing alternatives with better conditionality and diversity but less training effort. 1

Recently, large-scale datasets have vastly facilitated the development in nearly all domains of Natural Language Processing.However, there is currently no cross-task dataset in NLP, which hinders the development of multi-task learning.We propose MATINF, the first jointly labeled large-scale dataset for classification, question answering and summarization.MAT-INF contains 1.07 million question-answer pairs with human-labeled categories and usergenerated question descriptions.Based on such rich information, MATINF is applicable for three major NLP tasks, including classification, question answering, and summarization.We benchmark existing methods and a novel multi-task baseline over MATINF to inspire further research.Our comprehensive comparison and experiments over MATINF and other datasets demonstrate the merits held by MAT-INF. 1

Intimacy is a fundamental aspect of how we relate to others in social settings.Language encodes the social information of intimacy through both topics and other more subtle cues (such as linguistic hedging and swearing).Here, we introduce a new computational framework for studying expressions of the intimacy in language with an accompanying dataset and deep learning model for accurately predicting the intimacy level of questions (Pearson's r=0.87).Through analyzing a dataset of 80.5M questions across social media, books, and films, we show that individuals employ interpersonal pragmatic moves in their language to align their intimacy with social settings.Then, in three studies, we further demonstrate how individuals modulate their intimacy to match social norms around gender, social distance, and audience, each validating key findings from studies in social psychology.Our work demonstrates that intimacy is a pervasive and impactful social dimension of language.

In recent years, with the prevalence of social media and smart devices, people causally reveal their locations such as shops, hotels, and restaurants in their tweets. Recognizing and linking such fine-grained location mentions to well-defined location profiles are beneficial for retrieval and recommendation systems. In this paper, we propose DLocRL, a new deep learning pipeline for fine-grained location recognition and linking in tweets, and verify its effectiveness on a real-world Twitter dataset.

How to generate relevant and informative responses is one of the core topics in response generation area.Following the task formulation of machine translation, previous works mainly consider response generation task as a mapping from a source sentence to a target sentence.To realize this mapping, existing works tend to design intuitive but complex models.However, the relevant information existed in large dialogue corpus is mainly overlooked.In this paper, we propose Sequence to Sequence with Prototype Memory Network (S2SPMN) to exploit the relevant information provided by the large dialogue corpus to enhance response generation.Specifically, we devise two simple approaches in S2SPMN to select the relevant information (named prototypes) from the dialogue corpus.These prototypes are then saved into prototype memory network (PMN).Furthermore, a hierarchical attention mechanism is devised to extract the semantic information from the PMN to assist the response generation process.Empirical studies indicate the advantage of our model over several classical and strong baselines.