Our ability to continuously acquire, organize, and leverage knowledge is a key feature of human intelligence that AI systems must approximate to unlock their full potential. Given the challenges in continual learning with large language models (LLMs), retrieval-augmented generation (RAG) has become the dominant way to introduce new information. However, its reliance on vector retrieval hinders its ability to mimic the dynamic and interconnected nature of human long-term memory. Recent RAG approaches augment vector embeddings with various structures like knowledge graphs to address some of these gaps, namely sense-making and associativity. However, their performance on more basic factual memory tasks drops considerably below standard RAG. We address this unintended deterioration and propose HippoRAG 2, a framework that outperforms standard RAG comprehensively on factual, sense-making, and associative memory tasks. HippoRAG 2 builds upon the Personalized PageRank algorithm used in HippoRAG and enhances it with deeper passage integration and more effective online use of an LLM. This combination pushes this RAG system closer to the effectiveness of human long-term memory, achieving a 7% improvement in associative memory tasks over the state-of-the-art embedding model while also exhibiting superior factual knowledge and sense-making memory capabilities. This work paves the way for non-parametric continual learning for LLMs. Code and data are available at https://github.com/OSU-NLP-Group/HippoRAG.

Retrieval-augmented generation (RAG) systems combine large language models (LLMs) with external knowledge retrieval, making them highly effective for knowledge-intensive tasks. A crucial but often under-explored component of these systems is the reranker, which refines retrieved documents to enhance generation quality and explainability. The challenge of selecting the optimal number of documents (k) remains unsolved: too few may omit critical information, while too many introduce noise and inefficiencies. Although recent studies have explored LLM-based rerankers, they primarily leverage internal model knowledge and overlook the rich supervisory signals that LLMs can provide, such as using response quality as feedback for optimizing reranking decisions. In this paper, we propose DynamicRAG, a novel RAG framework where the reranker dynamically adjusts both the order and number of retrieved documents based on the query. We model the reranker as an agent optimized through reinforcement learning (RL), using rewards derived from LLM output quality. Across seven knowledge-intensive datasets, DynamicRAG demonstrates superior performance, achieving state-of-the-art results.

The field of open relation extraction (ORE) has recently observed significant advancement thanks to the growing capability of large language models (LLMs).Nevertheless, challenges persist when ORE is performed on specific topics.Existing methods give suboptimal results in five dimensions: factualness, topic relevance, informativeness, coverage, and uniformity.To improve topic-oriented ORE, we propose a zero-shot approach called Pri-ORE: Open Relation Extraction with a Priori seed generation.PriORE leverages the builtin knowledge of LLMs to maintain a dynamic seed relation dictionary for the topic.The dictionary is initialized by seed relations generated from topic-relevant entity types and expanded during contextualized ORE.PriORE then reduces the randomness in generative ORE by converting it to a more robust relation classification task.Experiments show the approach empowers better topic-oriented control over the generated relations and thus improves ORE performance along the five dimensions, especially on specialized and narrow topics. Dimension Extracted Relation Explanation Factualness gives toWrong given text and topic Relevance is exposed to Correct given text but not directly relevant to topic Informativeness form compound with Correct but conceptually too general given text and topic Coverage form powdery magnesium oxide Correct but too specific, covering too few instances Uniformity {be oxidized by, give electrons to, . . .} Multiple correct expressions extracted for the same relation

Relation extraction (RE) aims to identify semantic relationships between entities within text.Despite considerable advancements, existing models predominantly require extensive annotated training data, which is both costly and labor-intensive to collect.Moreover, these models often struggle to adapt to new or unseen relations.Few-shot learning, aiming to lessen annotation demands, typically provides incomplete and biased supervision for target relations, leading to degraded and unstable performance.To accurately and explicitly describe relation semantics while minimizing annotation demands, we explore the definition only zero-shot RE setting where only relation definitions expressed in natural language are used to train a RE model.We introduce REPAL, comprising three stages: (1) We leverage large language models (LLMs) to generate initial seed instances from relation definitions and an unlabeled corpus.(2) We fine-tune a bidirectional Small Language Model (SLM) with initial seeds to learn relations for the target domain.(3) We expand pattern coverage and mitigate bias from initial seeds by integrating feedback from the SLM's predictions on the unlabeled corpus and the synthesis history.To accomplish this, we leverage the multi-turn conversation ability of LLMs to generate new instances in follow-up dialogues, informed by both the feedback and synthesis history.Studies reveal that definition-oriented seed synthesis enhances pattern coverage whereas indiscriminately increasing seed quantity leads to performance saturation.Experiments on two datasets show REPAL significantly improved cost-effective zero-shot performance by large margins.

Massive amount of unstructured text data are generated daily, ranging from news articles to scientific papers. How to mine structured knowledge from the text data remains a crucial research question. Recently, large language models (LLMs) have shed light on the text mining field with their superior text understanding and instruction-following ability. There are typically two ways of utilizing LLMs: fine-tune the LLMs with human-annotated training data, which is labor intensive and hard to scale; prompt the LLMs in a zero-shot or few-shot way, which cannot take advantage of the useful information in the massive text data. Therefore, it remains a challenge on automated mining of structured knowledge from massive text data in the era of large language models.

Geospatial topological relation extraction (GeoTopoRE) aims to extract topological relations between named geospatial entities (i.e., geo-entities) in text. It is a domain-specific relation extraction (RE) task essential in geospatial knowledge graph construction and spatial reasoning. Unlike general-purpose RE, which primarily depends on semantic and syntactic cues, GeoTopoRE requires integrating geometric knowledge about geo-entities. This is essential for accurately capturing or inferring the complex geospatial relationships among entities. GeoTopoRE is not studied systematically and lacks dedicated datasets for evaluation, posing significant challenges to developing and assessing effective models. This study presents two major contributions: (i) the introduction of a high-quality, human-labeled dataset WikiTopo for the GeoTopoRE task, and (ii) a novel framework GeoWISE designed to adapt existing RE models to the GeoTopoRE task, With Integrated Semantic and External geospatial domain knowledge. We leverage coarse-to-fine-grained natural language inference (NLI) to align externally sourced knowledge with the semantic text context, enhanced by geospatial expertise. This integrated knowledge is then conveyed to language models as geospatial cues, enabling a nuanced understanding of topological relations. Empirical results demonstrate the efficacy of our framework in few-shot settings, showing significant and consistent improvements in the GeoTopoRE task for diverse state-of-the-art RE models.

Traditionally, offline datasets have been used to evaluate task-oriented dialogue (TOD) models. These datasets lack context awareness, making them suboptimal benchmarks for conversational systems. In contrast, user-agents, which are context-aware, can simulate the variability and unpredictability of human conversations, making them better alternatives as evaluators. Prior research has utilized large language models (LLMs) to develop user-agents. Our work builds upon this by using LLMs to create user-agents for the evaluation of TOD systems. This involves prompting an LLM, using in-context examples as guidance, and tracking the user-goal state. Our evaluation of diversity and task completion metrics for the user-agents shows improved performance with the use of better prompts. Additionally, we propose methodologies for the automatic evaluation of TOD models within this dynamic framework. We make our code publicly available11https://github.com/TaahaKazi/user-agent

Today a tremendous amount of geospatial knowledge is hidden in massive volumes of text data. To facilitate flexible and powerful geospatial analysis and applications, we introduce a new architecture: geospatial knowledge hypercube, a multi-scale, multidimensional knowledge structure that integrates information from geospatial dimensions, thematic themes and diverse application semantics, extracted and computed from spatial-related text data. To construct such a knowledge hypercube, weakly supervised language models are leveraged for automatic, dynamic and incremental extraction of heterogeneous geospatial data, thematic themes, latent connections and relationships, and application semantics, through combining a variety of information from unstructured text, structured tables, and maps. The hypercube lays a foundation for many knowledge discovery and in-depth spatial analysis, and other advanced applications. We have deployed a prototype web application of proposed geospatial knowledge hypercube for public access at: https://hcwebapp.cigi.illinois.edu/.