Demo Data from SIGIR 2026 / reviewers in the wild / expert
Michelle Wastl
dblp:366/7124
· DBLP profile ↗
2ranked-venue papers
1as first author
2since 2021 · last 2026
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 2 · 1 first-author · 2 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
2 papers |
Language models and text generation · 39% Trustworthy machine learning · 30% Information extraction and text analysis · 30% | |
| Interdisciplinary, comprehensive, and emerging computing
1 paper |
Computational social science and digital humanities · 100% |
Topics — the 3 heaviest of 5, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Natural language and speech › Language models and text generation › prompting
prompt engineering |
1.0 | 1 | 2026 | It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief · ACL (1) 2026 |
Machine learning › Trustworthy machine learning
robustness |
1.0 | 1 | 2026 | It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief · ACL (1) 2026 |
Natural language and speech › Language models and text generation › large language model evaluation › large language model benchmarking
LLM evaluation benchmark |
0.3 | 1 | 2026 | SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents · ACL (1) 2026 |
Methods — techniques the papers use, named apart from their topics
large language model · 2.0fine-tuning · 2.0encoder model · 2.0statistical analysis · 1.0benchmark construction · 1.0
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of BeliefabstractUsers frequently express their beliefs to large language models (LLMs).In some situations, it is ideal for the LLM to accept this contextual information as true, while in others, it is ideal to stick to prior knowledge.Users' expressions of belief (EoBs) can take linguistically diverse forms-using presuppositions, evidential and certainty markers, or varied toneseach of which may have a different persuasiveness over the LLMs.We introduce a benchmark to systematically evaluate how different EoBs affect whether models follow context versus prior knowledge.We propose a typology grounded in four linguistically motivated dimensions: form, evidentiality, epistemic stance, and tone, spanning 19 fine-grained types.By pairing these EoBs with world knowledge facts, we generate controlled EoB-query pairs that isolate the effect of linguistic variation.We use our benchmark to evaluate 18 LLMs that differ in architecture (Llama3, Qwen3, Gemma3), scale (1B-30B parameters), and training stages (base vs instruct).We identify meaningful variations in response behavior across these axes: For example, bigger models and instruction models tend to be less context-following than smaller models and base models.We further identify specific EoBs that statistically significantly persuade LMs more consistently than others.These systematic patterns in how linguistic framing affects LLM context integration serve to evaluate model robustness and inform best practices for prompt engineering.We publicly release code and data used in this project. Kevin Du, Clara Kümpel, Michelle Wastl, Alex Warstadt |
ACL (1) | 3 |
| 2026 | SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related DocumentsabstractRecognizing semantic differences across documents is crucial for text generation evaluation and content alignment, especially in crosslingual settings.However, as a standalone task, it has received little attention.We address this by introducing SwissGov-RSD, the first naturalistic, document-level, cross-lingual dataset for semantic difference recognition.It encompasses a total of 224 multi-parallel documents in English-German, English-French, and English-Italian with token-level difference annotations by human annotators.We evaluate a variety of open-source and closed-source large language models as well as encoder models across different fine-tuning settings on this new benchmark.Our results show that current automatic approaches perform poorly compared to their performance on monolingual, sentence-level, and synthetic benchmarks, revealing a considerable gap for both LLMs and encoder models.We make our code and dataset publicly available 1 . Michelle Wastl, Jannis Vamvas, Rico Sennrich |
ACL (1) | 1 |