Information extraction (IE) needs vary over time, where a flexible information extraction (IE) system can be useful. Despite this, existing IE systems are either fully supervised, requiring expensive human annotations, or fully unsupervised, extracting information that often do not cater to user’s needs. To address these issues, we formally introduce the task of “IE on-the-fly”, and address the problem using our proposed Adaptive IE framework that uses human-in-the-loop refinement to adapt to changing user questions. Through human experiments on three diverse datasets, we demonstrate that Adaptive IE is a domain-agnostic, responsive, efficient framework for helping users access useful information while quickly reorganizing information in response to evolving information needs.

Large language model (LLM) agents have evolved to intelligently process information, make decisions, and interact with users or tools. A key capability is the integration of long-term memory capabilities, enabling these agents to draw upon historical interactions and knowledge. However, the growing memory size and need for semantic structuring pose significant challenges. In this work, we propose an autonomous memory augmentation approach, MemInsight, to enhance semantic data representation and retrieval mechanisms. By leveraging autonomous augmentation to historical interactions, LLM agents are shown to deliver more accurate and contextualized responses. We empirically validate the efficacy of our proposed approach in three task scenarios; conversational recommendation, question answering and event summarization. On the LLM-REDIAL dataset, MemInsight boosts persuasiveness of recommendations by up to 14%. Moreover, it outperforms a RAG baseline by 34% in recall for LoCoMo retrieval. Our empirical results show the potential of MemInsight to enhance the contextual performance of LLM agents across multiple tasks.

Designing effective agentic systems requires the seamless composition and integration of agents, tools, and models within dynamic and uncertain environments. Most existing methods rely on static, semantic retrieval approaches for tool or agent discovery. However, effective reuse and composition of existing components remain challenging due to incomplete capability descriptions and the limitations of retrieval methods. Component selection suffers because the decisions are not based on capability, cost, and real-time utility. To address these challenges, we introduce a structured, automated framework for agentic system composition that is inspired by the knapsack problem. Our framework enables a composer agent to systematically identify, select, and assemble an optimal set of agentic components by jointly considering performance, budget constraints, and compatibility. By dynamically testing candidate components and modeling their utility in real-time, our approach streamlines the assembly of agentic systems and facilitates scalable reuse of resources. Empirical evaluation with Claude 3.5 Sonnet across five benchmarking datasets shows that our online-knapsack-based composer consistently lies on the Pareto frontier, achieving higher success rates at significantly lower component costs compared to our baselines. In the single-agent setup, the online knapsack composer shows a success rate improvement of up to 31.6\% in comparison to the retrieval baselines. In multi-agent systems, the online knapsack composer increases success rate from 37\% to 87\% when agents are selected from an agent inventory of 100+ agents. The substantial performance gap confirms the robust adaptability of our method across diverse domains and budget constraints.

Neural coreference resolution models trained on one dataset may not transfer to new, lowresource domains.Active learning mitigates this problem by sampling a small subset of data for annotators to label.While active learning is well-defined for classification tasks, its application to coreference resolution is neither well-defined nor fully understood.This paper explores how to actively label coreference, examining sources of model uncertainty and document reading costs.We compare uncertainty sampling strategies and their advantages through thorough error analysis.In both synthetic and human experiments, labeling spans within the same document is more effective than annotating spans across documents.The findings contribute to a more realistic development of coreference resolution models.

Active learning strives to reduce annotation costs by choosing the most critical examples to label.Typically, the active learning strategy is contingent on the classification model.For instance, uncertainty sampling depends on poorly calibrated model confidence scores.In the cold-start setting, active learning is impractical because of model instability and data scarcity.Fortunately, modern NLP provides an additional source of information: pretrained language models.The pre-training loss can find examples that surprise the model and should be labeled for efficient fine-tuning.Therefore, we treat the language modeling loss as a proxy for classification uncertainty.With BERT, we develop a simple strategy based on the masked language modeling loss that minimizes labeling costs for text classification.Compared to other baselines, our approach reaches higher accuracy within less sampling iterations and computation time.

Cross-lingual word embeddings transfer knowledge between languages: models trained on high-resource languages can predict in low-resource languages.We introduce CLIME, an interactive system to quickly refine cross-lingual word embeddings for a given classification problem.First, CLIME ranks words by their salience to the downstream task.Then, users mark similarity between keywords and their nearest neighbors in the embedding space.Finally, CLIME updates the embeddings using the annotations.We evaluate CLIME on identifying health-related text in four low-resource languages: Ilocano, Sinhalese, Tigrinya, and Uyghur.Embeddings refined by CLIME capture more nuanced word semantics and have higher test accuracy than the original embeddings.CLIME often improves accuracy faster than an active learning baseline and can be easily combined with active learning to improve results.

Multilingual topic models can reveal patterns in cross-lingual document collections. However, existing models lack speed and interactivity, which prevents adoption in everyday corpora exploration or quick moving situations (e.g., natural disasters, political instability). First, we propose a multilingual anchoring algorithm that builds an anchor-based topic model for documents in different languages. Then, we incorporate interactivity to develop MTAnchor (Multilingual Topic Anchors), a system that allows users to refine the topic model. We test our algorithms on labeled English, Chinese, and Sinhalese documents. Within minutes, our methods can produce interpretable topics that are useful for specific classification tasks.