Junlang Qian, Zixiao Zhu, Hanzhang Zhou, Zijian Feng, Zepeng Zhai, Kezhi Mao. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Zixiao Zhu, Zijian Feng, Hanzhang Zhou, Junlang Qian, Kezhi Mao. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Pruning is a widely used technique to reduce the size and inference cost of large language models (LLMs), but it often causes performance degradation. To mitigate this, existing restoration methods typically employ parameter-efficient fine-tuning (PEFT), such as LoRA, to recover the pruned model's performance. However, most PEFT methods are designed for dense models and overlook the distinct properties of pruned models, often resulting in suboptimal recovery. In this work, we propose a targeted restoration strategy for pruned models that restores performance while preserving their low cost and high efficiency. We observe that pruning-induced information loss is reflected in attention activations, and selectively reintroducing components of this information can significantly recover model performance. Based on this insight, we introduce RestoreLCC (Restoring Pruned LLMs via Lost Component Compensation), a plug-and-play method that contrastively probes critical attention heads via activation editing, extracts lost components from activation differences, and finally injects them back into the corresponding pruned heads for compensation and recovery. RestoreLCC is compatible with structured, semi-structured, and unstructured pruning schemes. Extensive experiments demonstrate that RestoreLCC consistently outperforms state-of-the-art baselines in both general and task-specific performance recovery, without compromising the sparsity or inference efficiency of pruned models.

Magnetic soft robots have the potential to be used in biomedical applications, such as targeted drug delivery, minimally invasive surgery, and on-chip tissue manipulation, due to their untethered operation, rapid actuation, and physical adaptability. However, real-time control of these robots is challenging because of their inherent nonlinear dynamics, fabrication imperfections, and complex interactions with external magnetic fields. In this work, we present a model-free controller that uses Proximal Policy Optimization-collected experience and a hybrid architecture, EpisodicMemNet. A memory module returns stored actions for angle-matched states, while a multi-head network predicts actions when no match is found. In experimental validation on two magnetic soft robots, a four-legged dual-stem H-frame (6 × 10 mm) and a three-legged asymmetric variant (8 × 11 mm), an adaptive two-tier memory maintained a median lookup time of approximately 1.2 ms, and EpisodicMemNet achieved 87.4% balanced accuracy and 93.5% top-2 accuracy while sustaining 2-Hz real-time control. Furthermore, user-guided tasks such as target navigation, obstacle avoidance, and ramp climbing confirmed reliable performance and adaptability despite the system’s nonlinearities and data sparsity. The proposed method thus not only overcomes the limitations of traditional simulation-based and model-specific approaches but also paves the way for scalable, experience-driven control solutions in soft robotic applications.

Controllable text generation (CTG) seeks to craft texts adhering to specific attributes, traditionally employing learning-based techniques such as training, fine-tuning, or prefix-tuning with attribute-specific datasets.These approaches, while effective, demand extensive computational and data resources.In contrast, some proposed learning-free alternatives circumvent learning but often yield inferior results, exemplifying the fundamental machine learning trade-off between computational expense and model efficacy.To overcome these limitations, we propose FreeCtrl, a learningfree approach that dynamically adjusts the weights of selected feedforward neural network (FFN) vectors to steer the outputs of large language models (LLMs).FreeCtrl hinges on the principle that the weights of different FFN vectors influence the likelihood of different tokens appearing in the output.By identifying and adaptively adjusting the weights of attributerelated FFN vectors, FreeCtrl can control the output likelihood of attribute keywords in the generated content.Extensive experiments on single-and multi-attribute control reveal that the learning-free FreeCtrl outperforms other learning-free and learning-based methods, successfully resolving the dilemma between learning costs and model performance 1 .

In this study, we explore in-context learning (ICL) in document-level event argument extraction (EAE) to alleviate the dependency on large-scale labeled data for this task.We introduce the Heuristic-Driven Link-of-Analogy (HD-LoA) prompting tailored for the EAE task.Specifically, we hypothesize and validate that LLMs learn task-specific heuristics from demonstrations in ICL.Building upon this hypothesis, we introduce an explicit heuristicdriven demonstration construction approach, which transforms the haphazard example selection process into a systematic method that emphasizes task heuristics.Additionally, inspired by the analogical reasoning of human, we propose the link-of-analogy prompting, which enables LLMs to process new situations by drawing analogies to known situations, enhancing their performance on unseen classes beyond limited ICL examples.Experiments show that our method outperforms existing prompting methods and few-shot supervised learning methods on document-level EAE datasets.Additionally, the HD-LoA prompting shows effectiveness in other tasks like sentiment analysis and natural language inference, demonstrating its broad adaptability 1 .

Prompts play a crucial role in guiding the responses of Large Language Models (LLMs). However, the intricate role of individual tokens in prompts, known as input saliency, in shaping the responses remains largely underexplored. Existing saliency methods either misalign with LLM generation objectives or rely heavily on linearity assumptions, leading to potential inaccuracies. To address this, we propose Token Distribution Dynamics (TDD), an elegantly simple yet remarkably effective approach to unveil and manipulate the role of prompts in generating LLM outputs. TDD leverages the robust interpreting capabilities of the language model head (LM head) to assess input saliency. It projects input tokens into the embedding space and then estimates their significance based on distribution dynamics over the vocabulary. We introduce three TDD variants: forward, backward, and bidirectional, each offering unique insights into token relevance. Extensive experiments reveal that the TDD surpasses state-of-the-art baselines with a big margin in elucidating the causal relationships between prompts and LLM outputs. Beyond mere interpretation, we apply TDD to two prompt manipulation tasks for controlled text generation: zero-shot toxic language suppression and sentiment steering. Empirical results underscore TDD's proficiency in identifying both toxic and sentimental cues in prompts, subsequently mitigating toxicity or modulating sentiment in the generated content.

Large language models (LLMs) have demonstrated impressive capabilities in various tasks using the in-context learning (ICL) paradigm. However, their effectiveness is often compromised by inherent bias, leading to prompt brittleness—sensitivity to design settings such as example selection, order, and prompt formatting. Previous studies have addressed LLM bias through external adjustment of model outputs, but the internal mechanisms that lead to such bias remain unexplored. Our work delves into these mechanisms, particularly investigating how feedforward neural networks (FFNs) and attention heads result in the bias of LLMs. By Interpreting the contribution of individual FFN vectors and attention heads, we identify the biased LLM components that skew LLMs' prediction toward specific labels. To mitigate these biases, we introduce UniBias, an inference-only method that effectively identifies and eliminates biased FFN vectors and attention heads. Extensive experiments across 12 NLP datasets demonstrate that UniBias significantly enhances ICL performance and alleviates prompt brittleness of LLMs.

Learning and reusing open source API libraries remain a time consuming process due to the documentation quality and the knowledge gap between API providers and users. Some researchers and API providers have found that the development tasks would narrow the knowledge gap and meet the needs of busy developers. To our knowledge, there is no existing work to generating task oriented API documents. In this paper, we propose an automatic approach to generating task oriented API learning guide. The guide is organized by a hierarchical task list. We integrate the natural language processing techniques with an evidence-based filtering pipeline in our approach. We also employ a graph-based clustering procedure to generate a three-layer task list. Furthermore, we define the normal form of the task phrases as the metadata in our approach. The approach has been implemented as a tool, APITasks. We used it to generate the API documents for four libraries. In an empirical study, we evaluate the accuracy and completeness of our approach with the manually created benchmarks. The results affirm the capability of our approach.

Lack of effective usage examples in API documents has been proven to be a great obstacle to API learning. To deal with this issue, several approaches have been proposed to automatically extract usage examples from client code or related web pages, which are unfortunately not available for newly released API libraries. In this paper, we propose a novel approach to mining API usage examples from test code. Although test code can be a good source of usage examples, the issue of multiple test scenarios might lead to repetitive and interdependent API usages in a test method, which make it complicated and difficult to extract API usage examples. To address this issue, we study the JUnit test code and summarize a set of test code patterns. We employ a code pattern based heuristic slicing approach to separate test scenarios into code examples. Then we cluster the similar usage examples for recommendation. An evaluation on four open source software libraries demonstrates that the accuracy of our approach is much higher than the state-of-art approach eXoaDoc on test code. Furthermore, we have developed an Eclipse plug in tool Use Tec.

Usage examples have been shown very helpful for API learning in software reuse. Nowadays, many approaches have been proposed to automatically extract usage examples from client code or web pages for API users. However, they overlooked the benefit of API developers in example publishing and few works paid attention to help API developers to generate usage examples automatically. In this paper, we proposed an approach to generate API-usage example based on test code before the project are released. It analyzed which parts in test code are important for indicating API-usage and summarized some test code patterns, then a heuristic slice algorithm are proposed to extract referential test code as API-usage example based on these patterns. In the experiments, we gave some case studies on the commons-lang3 open source software library. It proved that our approach can provide good assistance for developers in APIs usage example generation.