Quyu Kong, Xu Zhang, Zhenyu Yang, Nolan Gao, Chen Liu, Panrong Tong, Chenglin Cai, Hanzhang Zhou, Jianan Zhang, Liangyu Chen, Zhidan Liu, Steven Hoi, Yue Wang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Junlang Qian, Zixiao Zhu, Hanzhang Zhou, Zijian Feng, Zepeng Zhai, Kezhi Mao. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Zixiao Zhu, Zijian Feng, Hanzhang Zhou, Junlang Qian, Kezhi Mao. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Pruning is a widely used technique to reduce the size and inference cost of large language models (LLMs), but it often causes performance degradation. To mitigate this, existing restoration methods typically employ parameter-efficient fine-tuning (PEFT), such as LoRA, to recover the pruned model's performance. However, most PEFT methods are designed for dense models and overlook the distinct properties of pruned models, often resulting in suboptimal recovery. In this work, we propose a targeted restoration strategy for pruned models that restores performance while preserving their low cost and high efficiency. We observe that pruning-induced information loss is reflected in attention activations, and selectively reintroducing components of this information can significantly recover model performance. Based on this insight, we introduce RestoreLCC (Restoring Pruned LLMs via Lost Component Compensation), a plug-and-play method that contrastively probes critical attention heads via activation editing, extracts lost components from activation differences, and finally injects them back into the corresponding pruned heads for compensation and recovery. RestoreLCC is compatible with structured, semi-structured, and unstructured pruning schemes. Extensive experiments demonstrate that RestoreLCC consistently outperforms state-of-the-art baselines in both general and task-specific performance recovery, without compromising the sparsity or inference efficiency of pruned models.

Controllable text generation (CTG) seeks to craft texts adhering to specific attributes, traditionally employing learning-based techniques such as training, fine-tuning, or prefix-tuning with attribute-specific datasets.These approaches, while effective, demand extensive computational and data resources.In contrast, some proposed learning-free alternatives circumvent learning but often yield inferior results, exemplifying the fundamental machine learning trade-off between computational expense and model efficacy.To overcome these limitations, we propose FreeCtrl, a learningfree approach that dynamically adjusts the weights of selected feedforward neural network (FFN) vectors to steer the outputs of large language models (LLMs).FreeCtrl hinges on the principle that the weights of different FFN vectors influence the likelihood of different tokens appearing in the output.By identifying and adaptively adjusting the weights of attributerelated FFN vectors, FreeCtrl can control the output likelihood of attribute keywords in the generated content.Extensive experiments on single-and multi-attribute control reveal that the learning-free FreeCtrl outperforms other learning-free and learning-based methods, successfully resolving the dilemma between learning costs and model performance 1 .

In this study, we explore in-context learning (ICL) in document-level event argument extraction (EAE) to alleviate the dependency on large-scale labeled data for this task.We introduce the Heuristic-Driven Link-of-Analogy (HD-LoA) prompting tailored for the EAE task.Specifically, we hypothesize and validate that LLMs learn task-specific heuristics from demonstrations in ICL.Building upon this hypothesis, we introduce an explicit heuristicdriven demonstration construction approach, which transforms the haphazard example selection process into a systematic method that emphasizes task heuristics.Additionally, inspired by the analogical reasoning of human, we propose the link-of-analogy prompting, which enables LLMs to process new situations by drawing analogies to known situations, enhancing their performance on unseen classes beyond limited ICL examples.Experiments show that our method outperforms existing prompting methods and few-shot supervised learning methods on document-level EAE datasets.Additionally, the HD-LoA prompting shows effectiveness in other tasks like sentiment analysis and natural language inference, demonstrating its broad adaptability 1 .

Prompts play a crucial role in guiding the responses of Large Language Models (LLMs). However, the intricate role of individual tokens in prompts, known as input saliency, in shaping the responses remains largely underexplored. Existing saliency methods either misalign with LLM generation objectives or rely heavily on linearity assumptions, leading to potential inaccuracies. To address this, we propose Token Distribution Dynamics (TDD), an elegantly simple yet remarkably effective approach to unveil and manipulate the role of prompts in generating LLM outputs. TDD leverages the robust interpreting capabilities of the language model head (LM head) to assess input saliency. It projects input tokens into the embedding space and then estimates their significance based on distribution dynamics over the vocabulary. We introduce three TDD variants: forward, backward, and bidirectional, each offering unique insights into token relevance. Extensive experiments reveal that the TDD surpasses state-of-the-art baselines with a big margin in elucidating the causal relationships between prompts and LLM outputs. Beyond mere interpretation, we apply TDD to two prompt manipulation tasks for controlled text generation: zero-shot toxic language suppression and sentiment steering. Empirical results underscore TDD's proficiency in identifying both toxic and sentimental cues in prompts, subsequently mitigating toxicity or modulating sentiment in the generated content.

Large language models (LLMs) have demonstrated impressive capabilities in various tasks using the in-context learning (ICL) paradigm. However, their effectiveness is often compromised by inherent bias, leading to prompt brittleness—sensitivity to design settings such as example selection, order, and prompt formatting. Previous studies have addressed LLM bias through external adjustment of model outputs, but the internal mechanisms that lead to such bias remain unexplored. Our work delves into these mechanisms, particularly investigating how feedforward neural networks (FFNs) and attention heads result in the bias of LLMs. By Interpreting the contribution of individual FFN vectors and attention heads, we identify the biased LLM components that skew LLMs' prediction toward specific labels. To mitigate these biases, we introduce UniBias, an inference-only method that effectively identifies and eliminates biased FFN vectors and attention heads. Extensive experiments across 12 NLP datasets demonstrate that UniBias significantly enhances ICL performance and alleviates prompt brittleness of LLMs.

In document-level event argument extraction, an argument is likely to appear multiple times in different expressions in the document.The redundancy of arguments underlying multiple sentences is beneficial but is often overlooked.In addition, in event argument extraction, most entities are regarded as class "others", i.e.Universum class, which is defined as a collection of samples that do not belong to any class of interest.Universum class is composed of heterogeneous entities without typical common features.Classifiers trained by cross entropy loss could easily misclassify the Universum class because of their open decision boundary.In this paper, to make use of redundant event information underlying a document, we build an entity coreference graph with the graph2token module to produce a comprehensive and coreference-aware representation for every entity and then build an entity summary graph to merge the multiple extraction results.To better classify Universum class, we propose a new loss function to build classifiers with closed boundaries.Experimental results show that our model outperforms the previous state-of-the-art models by 3.35% in F1-score.

Thin-section magnetic resonance imaging (MRI) can provide higher resolution anatomical structures and more precise clinical information than thick-section images. However, thin-section MRI is not always available due to the imaging cost issue. In multicenter retrospective studies, a large number of data are often in thick-section manner with different section thickness. The lack of thin-section data and the difference in section thickness bring considerable difficulties in the study based on the image big data. In this article, we introduce DeepVolume, a two-step deep learning architecture to address the challenge of accurate thin-section MR image reconstruction. The first stage is the brain structure-aware network, in which the thick-section MR images in axial and sagittal planes are fused by a multitask 3-D U-net with prior knowledge of brain volume segmentation, which encourages the reconstruction result to have correct brain structure. The second stage is the spatial connection-aware network, in which the preliminary reconstruction results are adjusted slice-by-slice by a recurrent convolutional network embedding convolutional long short-term memory (LSTM) block, which enhances the precision of the reconstruction by utilizing the previously unassessed sagittal information. We used 305 paired brain MRI samples with thickness of 1.0 mm and 6.5 mm in this article. Extensive experiments illustrate that DeepVolume can produce the state-of-the-art reconstruction results by embedding more anatomical knowledge. Furthermore, considering DeepVolume as an intermediate step, the practical and clinical value of our method is validated by applying the brain volume estimation and voxel-based morphometry. The results show that DeepVolume can provide much more reliable brain volume estimation in the normalized space based on the thick-section MR images compared with the traditional solutions.