While language models demonstrate sophisticated syntactic capabilities, the extent to which their internal mechanisms align with crossconstructional principles studied in linguistics remains poorly understood.This study investigates whether models employ shared neural mechanisms across different syntactic constructions by applying causal interpretability methods at a granular level.Focusing on filler-gap dependencies and negative polarity item (NPI) licensing, we utilize activation patching to identify the functional roles of specific attention heads and MLP blocks.Our results reveal a highly localized and shared mechanism for filler-gap dependencies located in the early to middle layers, whereas NPI processing exhibits no such unified mechanism.Furthermore, we find that these mechanisms identified by activation patching generalize to out-of-distribution, while distributed alignment search, a supervised interpretability method, is susceptible to overfitting on narrow linguistic distributions.Finally, we validate our findings by demonstrating that the manipulation of the identified components improves model performance on acceptability judgment benchmarks. 1

Large language models (LLMs) exhibit social biases, prompting the development of various debiasing methods.However, debiasing methods may degrade the capabilities of LLMs.Previous research has evaluated the impact of bias mitigation primarily through tasks measuring general language understanding, which are often unrelated to social biases.In contrast, cultural commonsense is closely related to social biases, as both are rooted in social norms and values.The impact of bias mitigation on cultural commonsense in LLMs has not been well investigated.Considering this gap, we propose SOBACO (SOcial BiAs and Cultural cOmmonsense benchmark), a Japanese benchmark designed to evaluate social biases and cultural commonsense in LLMs in a unified format.We evaluate several LLMs on SOBACO to examine how debiasing methods affect cultural commonsense in LLMs.Our results reveal that the debiasing methods degrade the performance of the LLMs on the cultural commonsense task (up to 75% accuracy deterioration).These results highlight the importance of developing debiasing methods that consider the trade-off with cultural commonsense to improve fairness and utility of LLMs.Warning: This paper contains examples of social biases that can be offensive.

Ryoma Kumon, Hitomi Yanaka. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.