Large language models sometimes inadvertently reproduce passages that are copyrighted, exposing downstream applications to legal risk. Most existing studies for inference-time defences focus on surface-level token matching and rely on external blocklists or filters, which add deployment complexity and may overlook semantically paraphrased leakage. In this work, we reframe copyright infringement mitigation as intrinsic semantic-space control and introduce SCOPE, an inference-time method that requires no parameter updates or auxiliary filters. Specifically, the sparse autoencoder (SAE) projects hidden states into a high-dimensional, near-monosemantic space; benefiting from this representation, we identify a copyright-sensitive subspace and clamp its activations during decoding. Experiments on widely recognized benchmarks show that SCOPE mitigates copyright infringement without degrading general utility. Further interpretability analyses confirm that the isolated subspace captures high-level semantics.

Xunjian Yin, Sitao Cheng, Yuxi Xie, Xinyu Hu, Li Lin, Xinyi Wang, Liangming Pan, William Yang Wang, Xiaojun Wan. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

In NLG meta-evaluation, evaluation metrics are typically assessed based on their consistency with humans.However, we identify some limitations in traditional NLG meta-evaluation approaches, such as issues in handling human ratings and ambiguous selections of correlation measures, which undermine the effectiveness of meta-evaluation.In this work, we propose a dual-perspective NLG meta-evaluation framework that focuses on different evaluation capabilities, thereby providing better interpretability.In addition, we introduce a method of automatically constructing the corresponding benchmarks without requiring new human annotations.Furthermore, we conduct experiments with 16 representative LLMs as the evaluators based on our proposed framework, comprehensively analyzing their evaluation performance from different perspectives.

Large language models (LLMs) excel at various natural language processing tasks, but their tendency to generate hallucinations undermines their reliability. Existing hallucination detection methods leveraging hidden states predominantly focus on static and isolated representations, overlooking their dynamic evolution across layers, which limits efficacy. To address this limitation, we shift the focus to the hidden state update process and introduce a novel metric, the ICR Score (Information Contribution to Residual Stream), which quantifies the contribution of modules to the hidden states’ update. We empirically validate that the ICR Score is effective and reliable in distinguishing hallucinations. Building on these insights, we propose a hallucination detection method, the ICR Probe, which captures the cross-layer evolution of hidden states. Experimental results show that the ICR Probe achieves superior performance with significantly fewer parameters. Furthermore, ablation studies and case analyses offer deeper insights into the underlying mechanism of this method, improving its interpretability.

Large Language Models (LLMs) are increasingly integrated into academic peer review, prompting debates between full automation and purely human evaluation. Emerging evidence suggests optimal peer review leverages both human expertise and AI capabilities, and several major conferences have already adopted AI-assisted reviewing practices. However, effectively integrating these reviewers requires an aspect-based understanding of LLM vulnerabilities, clearly identifying specific dimensions where AI is most prone to error. Prior studies broadly caution against LLM biases but lack precise, aspect-specific insights necessary for informed human-AI partnerships in peer-review processes. We propose an aspect-guided, multi-level perturbation framework to systematically diagnose LLM weaknesses in automated peer review. By introducing targeted perturbations across key review components (papers, reviews, rebuttals) and evaluating impacts along critical quality dimensions (contribution, soundness, presentation, tone, completeness), our framework functions as a diagnostic tool: deviations from expected rating shifts after perturbation directly reveal specific LLM vulnerabilities. Our empirical analyses uncover recurring weaknesses, including misclassification of methodological flaws, disproportionate influence of strong rejection recommendations, inadequate responses to incomplete or negatively toned rebuttals, and misinterpretation of incorrect critiques as rigorous evaluations. These vulnerabilities consistently persist across diverse prompting strategies and a broad set of widely-used LLMs (e.g., GPT-4o, Gemini 2.0, LLaMA 3). This diagnostic framework provides granular insights into LLM limitations, empowering conference organizers to establish pragmatic, aspect-specific guidelines and enabling balanced, informed, and robust peer-review practices.

While large language models (LLMs) demonstrate remarkable capabilities across a wide range of tasks, they remain vulnerable to generating outputs that are potentially harmful.Red teaming, which involves crafting adversarial inputs to expose vulnerabilities, is a widely adopted approach for evaluating the robustness of these models.Prior studies have indicated that LLMs are susceptible to vulnerabilities exposed through multi-turn interactions as opposed to single-turn scenarios.Nevertheless, existing methods for multi-turn attacks mainly utilize a predefined dialogue pattern, limiting their effectiveness in realistic situations.Effective attacks require adaptive dialogue strategies that respond dynamically to the initial user prompt and the evolving context of the conversation.To address these limitations, we propose DAMON, a novel multi-turn jailbreak attack method.DAMON leverages Monte Carlo Tree Search (MCTS) to systematically explore multiturn conversational spaces, efficiently identifying sub-instruction sequences that induce harmful responses.We evaluate DAMON's efficacy across five LLMs and three datasets.Our experimental results show that DAMON can effectively induce undesired behaviors.

Mingqi Gao, Xinyu Hu, Li Lin, Xiaojun Wan. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Some prior work has shown that LLMs perform well in NLG evaluation for different tasks.However, we discover that LLMs seem to confuse different evaluation criteria, which reduces their reliability.For further verification, we first consider avoiding issues of inconsistent conceptualization and vague expression in existing NLG quality criteria themselves.So we summarize a clear hierarchical classification system for 11 common aspects with corresponding different criteria from previous studies involved.Inspired by behavioral testing, we elaborately design 18 types of aspect-targeted perturbation attacks for fine-grained analysis of the evaluation behaviors of different LLMs.We also conduct human annotations beyond the guidance of the classification system to validate the impact of the perturbations.Our experimental results reveal confusion issues inherent in LLMs, as well as other noteworthy phenomena, and necessitate further research and improvements for LLM-based evaluation. * Equal contribution. Prompt:Your task is to evaluate the summary written for a dialogue on the given criterion.

Chinese Spelling Check (CSC) aims to detect and correct error tokens in Chinese contexts, which has a wide range of applications. However, it is confronted with the challenges of insufficient annotated data and the issue that previous methods may actually not fully leverage the existing datasets. In this paper, we introduce our plug-and-play retrieval method with error-robust information for Chinese Spelling Check (RERIC), which can be directly applied to existing CSC models. The datastore for retrieval is built completely based on the training data, with elaborate designs according to the characteristics of CSC. Specifically, we employ multimodal representations that fuse phonetic, morphologic, and contextual information in the calculation of query and key during retrieval to enhance robustness against potential errors. Furthermore, in order to better judge the retrieved candidates, the n-gram surrounding the token to be checked is regarded as the value and utilized for specific reranking. The experiment results on the SIGHAN benchmarks demonstrate that our proposed method achieves substantial improvements over existing work.

The evaluation of natural language generation (NLG) tasks is a significant and longstanding research area.With the recent emergence of powerful large language models (LLMs), some studies have turned to LLM-based automatic evaluation methods, which demonstrate great potential to become a new evaluation paradigm following traditional string-based and modelbased metrics.However, despite the improved performance of existing methods, they still possess some deficiencies, such as dependency on references and limited evaluation flexibility.Therefore, in this paper, we meticulously construct a large-scale NLG evaluation corpus NLG-Eval with annotations from both human and GPT-4 to alleviate the lack of relevant data in this field.Furthermore, we propose Themis, an LLM dedicated to NLG evaluation, which has been trained with our designed multi-perspective consistency verification and rating-oriented preference alignment methods.Themis can conduct flexible and interpretable evaluations without references, and it exhibits superior evaluation performance on various NLG tasks, simultaneously generalizing well to unseen tasks and surpassing other evaluation models, including GPT-4.

Neural machine translation has achieved great success in the past few years with the help of transformer architectures and large-scale bilingual corpora.However, when the source text gradually grows into an entire document, the performance of current methods for documentlevel machine translation (DocMT) is less satisfactory.Although the context is beneficial to the translation in general, it is difficult for traditional methods to utilize such long-range information.Previous studies on DocMT have concentrated on extra contents such as multiple surrounding sentences and input instances divided by a fixed length.We suppose that they ignore the structure inside the source text, which leads to under-utilization of the context.In this paper, we present a more sound paragraph-to-paragraph translation mode and explore whether discourse structure can improve DocMT.We introduce several methods from different perspectives, among which our RST-Att model with a multi-granularity attention mechanism based on the RST parsing tree works best.The experiments show that our method indeed utilizes discourse information and performs better than previous work.

Previous studies have made great advances in RST discourse parsing through specific neural frameworks or features, but they usually split the parsing process into two subtasks and heavily depended on gold discourse segmentation. In this article, we introduce an end-to-end method for sentence-level RST discourse parsing via transforming it into a text-to-text generation task, which can also be simply applied to document-level parsing. Our method unifies the traditional two-stage parsing and generates the parsing tree directly from the input text through our constrained decoding and postprocessing algorithms, without requiring a complicated model. Moreover, the discourse segmentation can be simultaneously generated and extracted from the parsing tree. Experimental results on the RST Discourse Treebank demonstrate that our proposed method outperforms existing methods in both the tasks of discourse parsing and segmentation. We further carry out ablation studies and more targeted comparisons with traditional patterns to analyze our method in more detail. Considering the lack of annotated data in RST parsing, we also create high-quality augmented data and implement self-training, which further improves the performance of our method.