EDBT 2026 Demo / reviewers in the wild / expert
Kenan Tang
dblp:314/5791
· DBLP profile ↗
1ranked-venue papers
0as first author
1since 2021 · last 2025
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
1 paper |
Language models and text generation · 91% Trustworthy machine learning · 9% |
Topics — the 4 heaviest of 4, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Natural language and speech › Language models and text generation
large language model evaluation |
0.9 | 1 | 2025 | Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs · EMNLP 2025 |
Natural language and speech › Language models and text generation › large language model evaluation
LLM-as-a-judge |
0.9 | 1 | 2025 | Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs · EMNLP 2025 |
Natural language and speech › Language models and text generation › prompting
prompt sensitivity |
0.9 | 1 | 2025 | Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs · EMNLP 2025 |
Machine learning › Trustworthy machine learning
robustness |
0.3 | 1 | 2025 | Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs · EMNLP 2025 |
Methods — techniques the papers use, named apart from their topics
log-likelihood scoring · 0.9LLM-as-a-judge · 0.9
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMsabstractPrompt sensitivity, referring to the phenomenon where paraphrasing (i.e., repeating something written or spoken using different words) leads to significant changes in large language model (LLM) performance, has been widely accepted as a core limitation of LLMs.In this work, we revisit this issue and ask: Is the widely reported high prompt sensitivity truly an inherent weakness of LLMs, or is it largely an artifact of evaluation processes?To answer this question, we systematically evaluate 7 LLMs (e.g., GPT and Gemini family) across 6 benchmarks, including both multiple-choice and open-ended tasks on 12 diverse prompt templates.We find that much of the prompt sensitivity stems from heuristic evaluation methods, including log-likelihood scoring and rigid answer matching, which often overlook semantically correct responses expressed through alternative phrasings, such as synonyms or paraphrases.When we adopt LLM-as-a-Judge evaluations, we observe a substantial reduction in performance variance and a consistently higher correlation in model rankings across prompts.Our findings suggest that modern LLMs are more robust to prompt templates than previously believed, and that prompt sensitivity may be more an artifact of evaluation than a flaw in the models. Andong Hua, Kenan Tang, Chenhe Gu, Jindong Gu, Eric Wong 0001, Yao Qin 0001 |
EMNLP | 2 |