Demo Data from SIGIR 2026 / reviewers in the wild / expert
Zhuochun Li
dblp:362/0914
· DBLP profile ↗
2ranked-venue papers
0as first author
2since 2021 · last 2025
0009-0004-0772-9416ORCID · corroborated
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 since 2021Databases, data management, data science and information retrieval · 1 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Databases, data mining, and information retrieval
1 paper |
Information retrieval · 100% | |
| Artificial intelligence
1 paper |
Language models and text generation · 44% Knowledge representation and reasoning · 44% Trustworthy machine learning · 13% |
Topics — the 5 heaviest of 6, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Knowledge, reasoning and agents › Knowledge representation and reasoning
abstract reasoning |
0.9 | 1 | 2025 | Think Globally, Group Locally: Evaluating LLMs Using Multi-Lingual Word Grouping Games · EMNLP 2025 |
Natural language and speech › Language models and text generation
large language model evaluation |
0.9 | 1 | 2025 | Think Globally, Group Locally: Evaluating LLMs Using Multi-Lingual Word Grouping Games · EMNLP 2025 |
Information retrieval › reranking
document re-ranking |
0.9 | 1 | 2025 | Reason-to-Rank: Distilling Direct and Comparative Reasoning from Large Language Models for Document Reranking · SIGIR 2025 |
Information retrieval › ranking › ranking model
interpretable ranking |
0.9 | 1 | 2025 | Reason-to-Rank: Distilling Direct and Comparative Reasoning from Large Language Models for Document Reranking · SIGIR 2025 |
Information retrieval
ranking |
0.9 | 1 | 2025 | Reason-to-Rank: Distilling Direct and Comparative Reasoning from Large Language Models for Document Reranking · SIGIR 2025 |
Methods — techniques the papers use, named apart from their topics
large language model prompting · 0.9knowledge distillation · 0.9difficulty measurement · 0.9benchmark construction · 0.9
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | Think Globally, Group Locally: Evaluating LLMs Using Multi-Lingual Word Grouping GamesabstractLarge language models (LLMs) can exhibit biases in reasoning capabilities due to linguistic modality, performing better on tasks in one language versus another, even with similar content.Most previous works evaluate this through reasoning tasks where reliance on strategies or knowledge can ensure success, such as in commonsense or math tasks.However, abstract reasoning is vital to reasoning for everyday life, where people apply "out-of-the-box thinking" to identify and use patterns for solutions, without a reliance on formulaic approaches.Comparatively, little work has evaluated linguistic biases in this task type.In this paper, we propose a task inspired by the New York Times Connections: GLOBALGROUP, that evaluates models in an abstract reasoning task across several languages.We constructed a game benchmark with five linguistic backgrounds -English, Spanish, Chinese, Hindi, and Arabic -in both the native language and an English translation for comparison.We also proposed game difficulty measurements to evaluate models on games with similar difficulty, enabling a more controlled comparison, which is particularly important in reasoning evaluations.Through experimentation, we find English modalities largely lead to better performance in this abstract reasoning task, and performance disparities between open-and closed-source models. César Guerra-Solano, Zhuochun Li, Xiang Li 0069 |
EMNLP | 2 |
| 2025 | Reason-to-Rank: Distilling Direct and Comparative Reasoning from Large Language Models for Document RerankingabstractReranking documents in information retrieval often relies on black-box models that improve effectiveness but lack explainability. We introduce Reason-to-Rank (R2R), a novel framework that separates direct relevance reasoning from comparison reasoning to provide both direct and comparitive explanations. We first prompt a large language model to produce comprehensive rationales and a ranking order; then we distill both the ranking decisions and textual explanations into a smaller, open-source student model. Our approach not only improves retrieval performance, as demonstrated in MSMARCO, BEIR, and BRIGHT, but also provides interpretable justifications for why one document outranks another. We report NDCG@5 (and NDCG@10) for direct comparisons with prior work, and show that the distilled student model achieves competitive results while significantly reducing computational overhead. By unifying direct and comparative reasoning in a single pipeline, R2R bridges the gap between transparency and effectiveness in modern reranking systems. Yuelyu Ji, Zhuochun Li, Daqing He |
SIGIR | 2 |