Demo Data from SIGIR 2026 / reviewers in the wild / expert
Franklin Ma
dblp:420/4648
· DBLP profile ↗
1ranked-venue papers
0as first author
1since 2021 · last 2025
—ORCID · conflict
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
1 paper |
Knowledge representation and reasoning · 77% Language models and text generation · 23% |
Topics — the 2 heaviest of 2, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Knowledge, reasoning and agents › Knowledge representation and reasoning
metalinguistic reasoning |
0.9 | 1 | 2025 | LingGym: How Far Are LLMs from Thinking Like Field Linguists? · EMNLP 2025 |
Natural language and speech › Language models and text generation
large language model |
0.3 | 1 | 2025 | LingGym: How Far Are LLMs from Thinking Like Field Linguists? · EMNLP 2025 |
Methods — techniques the papers use, named apart from their topics
benchmark evaluation · 0.9
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | LingGym: How Far Are LLMs from Thinking Like Field Linguists?abstractThis paper introduces LINGGYM, a new benchmark that evaluates LLMs' capacity for metalinguistic reasoning using Interlinear Glossed Text (IGT) and grammatical descriptions extracted from 18 typologically diverse reference grammars.Unlike previous work that focuses on specific downstream tasks, we assess whether LLMs can generalize linguistic inference across low-resource languages and structures not seen during training.We present a controlled evaluation task: Word-Gloss Inference, in which the model must infer a missing word and gloss from context using varying levels of linguistic information (e.g., glosses, grammatical explanations, translations).Our results show that incorporating structured linguistic cues leads to consistent improvements in reasoning performance across all models.This work highlights both the promise and current limitations of using LLMs for typologically informed linguistic analysis and low-resource language documentation. Changbing Yang, Franklin Ma, Freda Shi |
EMNLP | 2 |