Taiwei Shi, Zhuoer Wang, Longqi Yang, Ying-Chun Lin, Zexue He, Mengting Wan, Pei Zhou, Sujay Kumar Jauhar, Sihao Chen, Shan Xia, Hongfei Zhang, Jieyu Zhao, Xiaofeng Xu, Xia Song, Jennifer Neville. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Steerability, or the ability of large language models (LLMs) to adapt outputs to align with diverse community-specific norms, perspectives, and communication styles, is critical for real-world applications but remains underevaluated.We introduce STEER-BENCH, a benchmark for assessing population-specific steering using contrasting Reddit communities.Covering 30 contrasting subreddit pairs across 19 domains, STEER-BENCH includes over 10,000 instruction-response pairs and validated 5,500 multiple-choice questions with corresponding silver labels to test alignment with diverse community norms.It systematically assesses how effectively LLMs understand community-specific instructions, their resilience to adversarial steering attempts, and their ability to accurately represent diverse cultural and ideological perspectives.Our evaluation of 13 popular LLMs using STEER-BENCH reveals that while human experts achieve an accuracy of 81% with silver labels, the bestperforming models reach only around 65% accuracy depending on the domain and configuration.Some models lag behind humanlevel alignment by over 15 percentage points, highlighting significant gaps in communitysensitive steerability.1Steered LLM Vanilla LLM Steered LLM Instruction: Why might some users decide to switch to Linux?Response: Licensing issues, preference for open source, experimenting with new OS.Instruction: Why might some users decide to switch to Linux?Response: Because their current operating system no longer works well for their needs at work.What motivates some Windows users to try Linux? A. Curiosity about open-source software.B. Frustration with Windows updates.C. Influence from tech industry trends.D. Desire to explore new GUI options.Steering towards r/linux Steering towards r/windows Open-ended Instruction with response from r/windows Open-ended Instruction with response from r/linux

Large Language Models (LLMs) possess the potential to exert substantial influence on public perceptions and interactions with information.This raises concerns about the societal impact that could arise if the ideologies within these models can be easily manipulated.In this work, we investigate how effectively LLMs can learn and generalize ideological biases from their instruction-tuning data.Our findings reveal a concerning vulnerability: exposure to only a small amount of ideologically driven samples significantly alters the ideology of LLMs.Notably, LLMs demonstrate a startling ability to absorb ideology from one topic and generalize it to even unrelated ones.The ease with which LLMs' ideologies can be skewed underscores the risks associated with intentionally poisoned training data by malicious actors or inadvertently introduced biases by data annotators.It also emphasizes the imperative for robust safeguards to mitigate the influence of ideological manipulations on LLMs. 1

Hyundong Cho, Shuai Liu, Taiwei Shi, Darpan Jain, Basem Rizk, Yuyang Huang, Zixun Lu, Nuan Wen, Jonathan Gratch, Emilio Ferrara, Jonathan May. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Taiwei Shi, Kai Chen, Jieyu Zhao. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Annotated data plays a critical role in Natural Language Processing (NLP) in training models and evaluating their performance.Given recent developments in Large Language Models (LLMs), models such as ChatGPT demonstrate zero-shot capability on many text-annotation tasks, comparable with or even exceeding human annotators.Such LLMs can serve as alternatives for manual annotation, due to lower costs and higher scalability.However, limited work has leveraged LLMs as complementary annotators, nor explored how annotation work is best allocated among humans and LLMs to achieve both quality and cost objectives.We propose CoAnnotating, a novel paradigm for Human-LLM co-annotation of unstructured texts at scale.Under this framework, we utilize uncertainty to estimate LLMs' annotation capability.Our empirical study shows CoAnnotating to be an effective means to allocate work from results on different datasets, with up to 21% performance improvement over random baseline.For code implementation, see https: //github.com/SALT-NLP/CoAnnotating.