Demo Data from SIGIR 2026 / reviewers in the wild / expert
Jane Lo
dblp:413/7478
· DBLP profile ↗
1ranked-venue papers
0as first author
1since 2021 · last 2025
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 since 2021Databases, data management, data science and information retrieval · 1 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
1 paper |
Language models and text generation · 33% Reinforcement learning · 33% Trustworthy machine learning · 33% | |
| Databases, data mining, and information retrieval
1 paper |
Information retrieval · 100% |
Topics — the 3 heaviest of 4, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Machine learning › Reinforcement learning
agent evaluation |
0.9 | 1 | 2025 | Evaluation and Benchmarking of LLM Agents: A Survey · KDD (2) 2025 |
Natural language and speech › Language models and text generation
LLM agents |
0.9 | 1 | 2025 | Evaluation and Benchmarking of LLM Agents: A Survey · KDD (2) 2025 |
Information retrieval › evaluation
benchmark |
0.3 | 1 | 2025 | Evaluation and Benchmarking of LLM Agents: A Survey · KDD (2) 2025 |
Methods — techniques the papers use, named apart from their topics
survey · 1.7
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | Evaluation and Benchmarking of LLM Agents: A SurveyabstractThe rise of LLM-based agents has opened new frontiers in AI applications, yet evaluating these agents remains a complex and underdeveloped area.This survey provides an in-depth overview of the emerging field of LLM agent evaluation, introducing a twodimensional taxonomy that organizes existing work along (1) evaluation objectives-what to evaluate, such as agent behavior, capabilities, reliability, and safety-and (2) evaluation process-how to evaluate, including interaction modes, datasets and benchmarks, metric computation methods, and tooling.In addition to taxonomy, we highlight enterprise-specific challenges, such as role-based access to data, the need for reliability guarantees, dynamic and longhorizon interactions, and compliance, which are often overlooked in current research.We also identify the future research directions, including holistic, more realistic, and scalable evaluation.This work aims to bring clarity to the fragmented landscape of agent evaluation and provide a framework for systematic assessment, enabling researchers and practitioners to evaluate LLM agents for real-world deployment. Mahmoud Mohammadi, Jane Lo, Wendy Yip |
KDD (2) | 3 |