Demo Data from SIGIR 2026 / reviewers in the wild / expert
Yishuai Qiu
dblp:389/8868
· DBLP profile ↗
1ranked-venue papers
0as first author
1since 2021 · last 2024
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
1 paper |
Language models and text generation · 87% Question answering and dialogue systems · 13% | |
| Interdisciplinary, comprehensive, and emerging computing
1 paper |
Medical and health informatics · 100% |
Topics — the 3 heaviest of 4, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Natural language and speech › Language models and text generation
large language model evaluation |
0.8 | 1 | 2024 | CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios · EMNLP 2024 |
Natural language and speech › Language models and text generation › large language model evaluation › domain-specific benchmark
medical benchmark |
0.8 | 1 | 2024 | CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios · EMNLP 2024 |
Natural language and speech › Question answering and dialogue systems
medical reasoning |
0.2 | 1 | 2024 | CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios · EMNLP 2024 |
Methods — techniques the papers use, named apart from their topics
expert-guided scenario design · 1.5
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2024 | CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical ScenariosabstractWith the proliferation of Large Language Models (LLMs) in diverse domains, there is a particular need for unified evaluation standards in Chinese clinical medical scenarios, where models need to be examined very thoroughly.We present CliMedBench, a comprehensive benchmark with 14 expert-guided core clinical scenarios specifically designed to assess the medical ability of LLMs across 7 pivot dimensions 1 .It comprises 33,735 questions derived from real-world medical reports of top-tier tertiary hospitals and authentic examination exercises.The reliability of this benchmark has been confirmed in several ways.Subsequent experiments with existing LLMs have led to the following findings: (i) Chinese medical LLMs underperform on this benchmark, especially where medical reasoning and factual consistency are vital, underscoring the need for advances in clinical knowledge and diagnostic accuracy.(ii) Several general-domain LLMs demonstrate substantial potential in medical clinics, while the limited input capacity of many medical LLMs hinders their practical use.These findings reveal both the strengths and limitations of LLMs in clinical scenarios and offer critical insights for medical research. Zetian Ouyang, Yishuai Qiu, Gerard de Melo, Ya Zhang 0002, Yanfeng Wang 0001, Liang He 0001 |
EMNLP | 2 |