Demo Data from VLDB 2026 / reviewers in the wild / expert
Adam Zahradník
dblp:425/8802
· DBLP profile ↗
1ranked-venue papers
1as first author
1since 2021 · last 2025
0009-0008-0106-5320ORCID · reported
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 first-author · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
1 paper |
Language models and text generation · 75% Information extraction and text analysis · 25% |
Topics — the 4 heaviest of 4, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Natural language and speech › Information extraction and text analysis › text classification
automated scoring |
0.9 | 1 | 2025 | Trojsten Benchmark: Evaluating LLM Problem-Solving in Slovak STEM Competition Problems · EMNLP 2025 |
Natural language and speech › Language models and text generation › evaluation of language models › multilingual evaluation
low-resource language evaluation |
0.9 | 1 | 2025 | Trojsten Benchmark: Evaluating LLM Problem-Solving in Slovak STEM Competition Problems · EMNLP 2025 |
Natural language and speech › Language models and text generation › evaluation of language models
multilingual evaluation |
0.9 | 1 | 2025 | Trojsten Benchmark: Evaluating LLM Problem-Solving in Slovak STEM Competition Problems · EMNLP 2025 |
Natural language and speech › Language models and text generation › evaluation of language models
reasoning evaluation |
0.9 | 1 | 2025 | Trojsten Benchmark: Evaluating LLM Problem-Solving in Slovak STEM Competition Problems · EMNLP 2025 |
Methods — techniques the papers use, named apart from their topics
rubric generation · 0.9LLM grading · 0.9
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | Trojsten Benchmark: Evaluating LLM Problem-Solving in Slovak STEM Competition ProblemsabstractLarge language models show promising performance on reasoning tasks, yet evaluation methods for low-resource languages remain limited, particularly for complex STEM problemsolving.We introduce Trojsten Benchmark, a Slovak-language dataset of 1,108 high-school competition problems with reference solutions across mathematics, physics, and programming, and a rubric-based LLM grading framework.Using GPT-4 to generate rubrics and grade solutions, we observe 1.05 average absolute deviation from human graders (5-point scale), while benchmarking GPT-3.5-Turbo,GPT-4, GPT-4o, and open-weight models (Llama 3, Phi-3).We quantify multistep reasoning performance by difficulty, show consistent underperformance on harder items, and demonstrate language sensitivity: accuracy drops on English translations of Slovak statements, evidencing challenges beyond translation.Trojsten Benchmark complements English-centric math datasets (e.g., MATH, GSM8K) by targeting open-response, rubric-gradable reasoning under low-resource linguistic framing.We release code and data to enable reproducible evaluation and humanaligned auto-grading for STEM in under-served languages. Adam Zahradník, Marek Suppa |
EMNLP | 1 |