Kenan Tang

dblp:314/5791 · DBLP profile ↗
← Back
1ranked-venue papers
0as first author
1since 2021 · last 2025
—ORCID · none

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 1 · 1 since 2021

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Artificial intelligence
1 paper
Language models and text generation · 91% Trustworthy machine learning · 9%

Topics — the 4 heaviest of 4, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Natural language and speech › Language models and text generation
large language model evaluation
0.912025
Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs · EMNLP 2025
Natural language and speech › Language models and text generation › large language model evaluation
LLM-as-a-judge
0.912025
Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs · EMNLP 2025
Natural language and speech › Language models and text generation › prompting
prompt sensitivity
0.912025
Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs · EMNLP 2025
Machine learning › Trustworthy machine learning
robustness
0.312025
Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs · EMNLP 2025

Methods — techniques the papers use, named apart from their topics

log-likelihood scoring · 0.9LLM-as-a-judge · 0.9
YearPublicationVenuePosition
2025 Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs
abstract
Prompt sensitivity, referring to the phenomenon where paraphrasing (i.e., repeating something written or spoken using different words) leads to significant changes in large language model (LLM) performance, has been widely accepted as a core limitation of LLMs.In this work, we revisit this issue and ask: Is the widely reported high prompt sensitivity truly an inherent weakness of LLMs, or is it largely an artifact of evaluation processes?To answer this question, we systematically evaluate 7 LLMs (e.g., GPT and Gemini family) across 6 benchmarks, including both multiple-choice and open-ended tasks on 12 diverse prompt templates.We find that much of the prompt sensitivity stems from heuristic evaluation methods, including log-likelihood scoring and rigid answer matching, which often overlook semantically correct responses expressed through alternative phrasings, such as synonyms or paraphrases.When we adopt LLM-as-a-Judge evaluations, we observe a substantial reduction in performance variance and a consistently higher correlation in model rankings across prompts.Our findings suggest that modern LLMs are more robust to prompt templates than previously believed, and that prompt sensitivity may be more an artifact of evaluation than a flaw in the models.
Andong Hua, Kenan Tang, Chenhe Gu, Jindong Gu, Eric Wong 0001, Yao Qin 0001
EMNLP2