Bingkang Shi, Jen-tse Huang, Luo Long, Tianyu Zong, Hongzhu Yi, Yuanxiang Wang, Songlin Hu, Xiaodan Zhang, Zhongjiang Yao. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Tianyu Zong, Rui Dai, Hongzhu Yi, Yuanxiang Wang, Zhenghao Zhang, Zhenyu Guan, Yujia Yang, Bingkang Shi, Yueyang Ding, Xiangxiang Chu, Kaikui Liu, Jungang Xu. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Unsupervised sentence embedding representation has become a hot research topic in natural language processing. As a tensor, sentence embedding has two critical properties: direction and norm. Existing works have been limited to constraining only the orientation of the samples' representations while ignoring the features of their module lengths. To address this issue, we propose a new training objective that optimizes the training of unsupervised contrastive learning by constraining the module length features between positive samples. We combine the training objective of Tensor's Norm Constraints with ensemble learning to propose a new Sentence Embedding representation framework, TNCSE. We evaluate seven semantic text similarity tasks, and the results show that TNCSE and derived models are the current state-of-the-art approach; in addition, we conduct extensive zero-shot evaluations, and the results show that TNCSE outperforms other baselines.

The social biases and unwelcome stereotypes revealed by pretrained language models are becoming obstacles to their application. Compared to numerous debiasing methods targeting word level, there has been relatively less attention on biases present at phrase level, limiting the performance of debiasing in discipline domains. In this paper, we propose an automatic multi-token debiasing pipeline called General Phrase Debiaser, which is capable of mitigating phrase-level biases in masked language models. Specifically, our method consists of a phrase filter stage that generates stereotypical phrases from Wikipedia pages as well as a model debias stage that can debias models at the multi-token level to tackle bias challenges on phrases. The latter searches for prompts that trigger model’s bias, and then uses them for debiasing. State-of-the-art results on standard datasets and metrics show that our approach can significantly reduce gender biases on both career and multiple disciplines, across models with varying parameter sizes.

As Large Language Models (LLMs) continue to enhance their capabilities in multi-agent collaborative applications, the unpredictability of the generative content risks has intensified. Particularly in ongoing interaction scenarios with users, it remains unclear whether there is generative content risk drift over time. In this context, "drift risk" refers to the trend of progressively intensified content risk that emerges during sustained adversarial interactions among LLM agents. Additionally, the high cost associated with constructing complex adversarial environments for agents impedes the transferability of current assessment methods for LLMs to multi-agent adversarial scenarios. In this paper, we introduce a low-cost and lightweight framework for assessing content risk drift of LLMs, named CRDA. This framework, bypassing the need for constructing complex adversarial environments, offers a method that integrates roles and responses memory to guide automatically multi-round adversarial interactions among LLM agents, that is, multiple agents as avatars of a single LLM. In this approach, LLM agents enable the analysis of content risk drift of this LLM. Moreover, we explore the impact of restricted roles and the unsafe content with negative viewpoints in responses memory on the content risk drift of LLMs. Considering the rapid advancement of Chinese LLM capabilities, this study selects real adversarial topics in Chinese and assesses content risk drift of five representative Chinese LLMs. The research finds that these LLMs exhibit significant content risk drift even after a certain safety alignment, showing an initial increase followed by a gradual decrease. As the adversarial process progresses, under restricted roles, agents more effectively breach the model's safety alignment, leading to content risk drift of the LLM. The content drift risk assessment can be quantified specifically by measuring the deterioration rate at which LLM agents deteriorate from positive to negative and analyzing the underlying trends during the automatically multi-round adversarial interactions. In restricted and general roles adversarial interactions, all agents of five Chinese LLMs exhibit an overall average increase of 31.5% and 16.38% in the cumulative deterioration rate respectively by the 10th round, compared to the baseline no-roles adversarial interactions. Finally, we hope that the framework and findings presented in this paper will offer valuable insights for research on safety alignment in LLM agents during adversarial processes.