Production LLM systems often rely on separate models for safety and other classificationheavy steps, increasing latency, VRAM footprint, and operational complexity.We instead reuse computation already paid for by the serving LLM: we train lightweight probes on its hidden states and predict labels in the same forward pass used for generation.We frame classification as representation selection over the full token×layer hidden-state tensor, rather than committing to a fixed token or fixed layer (e.g., first-token logits or final-layer pooling).To implement this, we introduce a two-stage aggregator that (i) summarizes tokens within each layer and (ii) aggregates across layer summaries to form a single representation for classification.We instantiate this template with direct pooling, a 100K-parameter scoring-attention gate, and a downcast multi-head self-attention (MHA) probe with up to 35M trainable parameters.Across safety and sentiment benchmarks our probes improve over logit-only reuse (e.g., MULI) and are competitive with substantially larger task-specific baselines, while preserving near-serving latency and avoiding the VRAM and latency costs of a separate guardmodel pipeline.Multi-backbone experiments on dense and mixture-of-experts architectures (Llama-3.2-3B,GPT-OSS-20B, Qwen3-30B-A3B) confirm that these findings generalize beyond a single model family.

Joaquin Polonuer, Lucas Vittor, Iñaki Arango, Ayush Noori, David A. Clifton, Luciano Del Corro, Marinka Zitnik. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Giovanni Franco Gabriel Marraffini, Andrés Cotton, Noe Fabian Hsueh, Axel Fridman, Juan Wisznia, Luciano Del Corro. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

We investigate post-OCR correction in a setting where we have access to different OCR views of the same document.The goal of this study is to understand if a pretrained language model (LM) can be used in an unsupervised way to reconcile the different OCR views such that their combination contains fewer errors than each individual view.This approach is motivated by scenarios in which unconstrained text generation for error correction is too risky.We evaluated different pretrained LMs on two datasets and found significant gains in realistic scenarios with up to 15% WER improvement over the best OCR view.We also show the importance of domain adaptation for post-OCR correction on out-of-domain documents.

This work introduces fact salience: The task of generating a machine-readable representation of the most prominent information in a text document as a set of facts.We also present SALIE, the first fact salience system.SALIE is unsupervised and knowledge agnostic, based on open information extraction to detect facts in natural language text, PageRank to determine their relevance, and clustering to promote diversity.We compare SALIE with several baselines (including positional, standard for saliency tasks), and in an extrinsic evaluation, with state-of-the-art automatic text summarizers.SALIE outperforms baselines and text summarizers showing that facts are an effective way to compress information.

The goal of Open Information Extraction (OIE) is to extract surface relations and their arguments from naturallanguage text in an unsupervised, domainindependent manner.In this paper, we propose MinIE, an OIE system that aims to provide useful, compact extractions with high precision and recall.MinIE approaches these goals by (1) representing information about polarity, modality, attribution, and quantities with semantic annotations instead of in the actual extraction, and (2) identifying and removing parts that are considered overly specific.We conducted an experimental study with several real-world datasets and found that MinIE achieves competitive or higher precision and recall than most prior systems, while at the same time producing shorter, semantically enriched extractions.Pinocchio believes that the hero Superman was not actually born on beautiful Krypton.OLLIE 1 (Pinocchio, believes that, the hero [...] beautiful Krypton) 2 (Superman, was not actually born on, beautiful Krypton) 3 (Superman, was not actually born on beau.Krypton in, the hero) ClausIE 4 (Pinocchio, believes, that the hero [...] beautiful K.) 5 (the hero Superman, was not born, on beautiful Krypton) 6 (the hero Superman, was not born, on beautiful Krypton actually) Stanford OIE No extractions MinIE-C(om-7 (Superman, was born actually on, beautiful Krypton) plete) A.: fact.(-[not], CT), attrib.(Pinocchio, +, PS [believes]) 8 (Superman, was born on, beautiful Krypton) A.: fact.(-[not], CT), attrib.(Pinocchio, +, PS [believes]) 9 (Superman, "is", hero) A.: fact.(+, CT)

We propose FINET, a system for detecting the types of named entities in short inputs-such as sentences or tweets-with respect to WordNet's super fine-grained type system.FINET generates candidate types using a sequence of multiple extractors, ranging from explicitly mentioned types to implicit types, and subsequently selects the most appropriate using ideas from word-sense disambiguation.FINET combats data scarcity and noise from existing systems: It does not rely on supervision in its extractors and generates training data for type selection from WordNet and other resources.FINET supports the most fine-grained type system so far, including types with no annotated training data.Our experiments indicate that FINET outperforms state-of-the-art methods in terms of recall, precision, and granularity of extracted types.

We propose CORE, a novel matrix factorization model that leverages contextual information for open relation extraction.Our model is based on factorization machines and integrates facts from various sources, such as knowledge bases or open information extractors, as well as the context in which these facts have been observed.We argue that integrating contextual information-such as metadata about extraction sources, lexical context, or type information-significantly improves prediction performance.Open information extractors, for example, may produce extractions that are unspecific or ambiguous when taken out of context.Our experimental study on a large real-world dataset indicates that CORE has significantly better prediction performance than state-ofthe-art approaches when contextual information is available.

Word-sense recognition and disambigua-tion (WERD) is the task of identifying word phrases and their senses in natural language text. Though it is well under-stood how to disambiguate noun phrases, this task is much less studied for verbs and verbal phrases. We present Werdy, a framework for WERD with particular focus on verbs and verbal phrases. Our framework first identifies multi-word ex-pressions based on the syntactic structure of the sentence; this allows us to recog-nize both contiguous and non-contiguous phrases. We then generate a list of can-didate senses for each word or phrase, us-ing novel syntactic and semantic pruning techniques. We also construct and lever-age a new resource of pairs of senses for verbs and their object arguments. Finally, we feed the so-obtained candidate senses into standard word-sense disambiguation (WSD) methods, and boost their precision and recall. Our experiments indicate that Werdy significantly increases the perfor-mance of existing WSD methods. 1

We propose ClausIE, a novel, clause-based approach to open information extraction, which extracts relations and their arguments from natural language text. ClausIE fundamentally differs from previous approaches in that it separates the detection of ``useful'' pieces of information expressed in a sentence from their representation in terms of extractions. In more detail, ClausIE exploits linguistic knowledge about the grammar of the English language to first detect clauses in an input sentence and to subsequently identify the type of each clause according to the grammatical function of its constituents. Based on this information, ClausIE is able to generate high-precision extractions; the representation of these extractions can be flexibly customized to the underlying application. ClausIE is based on dependency parsing and a small set of domain-independent lexica, operates sentence by sentence without any post-processing, and requires no training data (whether labeled or unlabeled). Our experimental study on various real-world datasets suggests that ClausIE obtains higher recall and higher precision than existing approaches, both on high-quality text as well as on noisy text as found in the web.