Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Hangyi Hao

dblp:410/3269 · DBLP profile ↗
← Back
1ranked-venue papers
0as first author
1since 2021 · last 2025
—ORCID · none

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 1 · 1 since 2021

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Interdisciplinary, comprehensive, and emerging computing
1 paper
Computing education · 100%
Artificial intelligence
1 paper
Language models and text generation · 100%

Topics — the 4 heaviest of 4, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Natural language and speech › Language models and text generation
code generation
0.912025
LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? · NeurIPS 2025
Computing education › programming education
competitive programming
0.912025
LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? · NeurIPS 2025
Computing education
large language model evaluation
0.912025
LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? · NeurIPS 2025
Computing education
programming education
0.912025
LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? · NeurIPS 2025

Methods — techniques the papers use, named apart from their topics

line-by-line failure analysis · 1.7benchmark annotation · 1.7
YearPublicationVenuePosition
2025 LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?
abstract
Recent reports claim that large language models (LLMs) now outperform elite humans in competitive programming. Drawing on knowledge from a group of medalists in international algorithmic contests, we revisit this claim, examining how LLMs differ from human experts and where limitations still remain. We introduce LiveCodeBench Pro, a benchmark composed of problems from Codeforces, ICPC, and IOI that are continuously updated to reduce the likelihood of data contamination. A team of Olympiad medalists annotates every problem for algorithmic categories and conducts a line-by-line analysis of failed model-generated submissions. Using this new data and benchmark, we find that frontier models still have significant limitations: without external tools, the best model achieves only 53\% pass@1 on medium-difficulty problems and 0\% on hard problems, domains where expert humans still excel. We also find that LLMs succeed at implementation-heavy problems but struggle with nuanced algorithmic reasoning and complex case analysis, often generating confidently incorrect justifications. High performance appears largely driven by implementation precision and tool augmentation, not superior reasoning. LiveCodeBench Pro thus highlights the significant gap to human grandmaster levels, while offering fine-grained diagnostics to steer future improvements in code-centric LLM reasoning.
Zihan Zheng, Zerui Cheng, Shang Zhou, Hansen He, Dongruixuan Li, Stanley Wei, Hangyi Hao, Jianzhu Yao, Peiyao Sheng, Zixuan Wang 0029, Wenhao Chai, Aleksandra Korolova, Peter Henderson 0002, Sanjeev Arora, Pramod Viswanath, Jingbo Shang, Saining Xie
NeurIPS9