Large language models (LLMs) often achieve high performance in native language identification (NLI) benchmarks by leveraging superficial contextual clues such as names, locations, and cultural stereotypes, rather than the underlying linguistic patterns indicative of native language (L1) influence.To improve robustness, previous work has instructed LLMs to disregard such clues.In this work, we demonstrate that such a strategy is unreliable and model predictions can be easily altered by misleading hints.To address this problem, we introduce an agentic NLI pipeline inspired by forensic linguistics, where specialized agents accumulate and categorize diverse linguistic evidence before an independent final overall assessment.In this final assessment, a goal-aware coordinating agent synthesizes all evidence to make the NLI prediction.On two benchmark datasets, our approach significantly enhances NLI robustness against misleading contextual clues and performance consistency compared to standard prompting methods. 1

Tannon Kew, Alison Chi, Laura Vásquez-Rodríguez, Sweta Agrawal, Dennis Aumiller, Fernando Alva-Manchego, Matthew Shardlow. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.