Demo Data from VLDB 2026 / reviewers in the wild / expert
Jianzhu Yao
dblp:317/0113
· DBLP profile ↗
2ranked-venue papers
1as first author
2since 2021 · last 2026
0009-0004-3854-9928ORCID · reported
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 since 2021Systems, architecture and hardware · 1 · 1 first-author · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Interdisciplinary, comprehensive, and emerging computing
1 paper |
Computing education · 100% | |
| Network and information security
1 paper |
Security and privacy of machine learning · 50% Hardware security and side channels · 50% | |
| Artificial intelligence
1 paper |
Language models and text generation · 100% |
Topics — the 6 heaviest of 6, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Security and privacy of machine learning › verifiable machine learning
model verification |
1.0 | 1 | 2026 | TAO: Tolerance-Aware Optimistic Verification for Floating-Point Neural Networks · EuroSys 2026 |
Hardware security and side channels
trusted execution environments |
1.0 | 1 | 2026 | TAO: Tolerance-Aware Optimistic Verification for Floating-Point Neural Networks · EuroSys 2026 |
Natural language and speech › Language models and text generation
code generation |
0.9 | 1 | 2025 | LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? · NeurIPS 2025 |
Computing education › programming education
competitive programming |
0.9 | 1 | 2025 | LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? · NeurIPS 2025 |
Computing education
large language model evaluation |
0.9 | 1 | 2025 | LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? · NeurIPS 2025 |
Computing education
programming education |
0.9 | 1 | 2025 | LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming? · NeurIPS 2025 |
Methods — techniques the papers use, named apart from their topics
line-by-line failure analysis · 1.7benchmark annotation · 1.7merkle tree · 1.0interactive dispute game · 1.0empirical percentile profiling · 1.0IEEE-754 error bounds · 1.0
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | TAO: Tolerance-Aware Optimistic Verification for Floating-Point Neural NetworksabstractNeural networks increasingly run on hardware outside the user's control (cloud GPUs, inference marketplaces, edge specialized accelerators) for both training and inference. Yet ML-as-a-Service reveals little about what actually ran or whether returned outputs faithfully reflect the intended inputs and models. Users lack recourse against service downgrades such as model swaps, quantization, graph rewrites, or discrepancies like altered advertisement embeddings. Verifying outputs is especially difficult because floating-point execution on heterogeneous accelerators is inherently non-deterministic. Existing approaches like zkML, deterministic replay, TEEs, and replication are either impractical for real floating-point neural networks or reintroduce the need to trust the vendor. We present TAO: a Tolerance-Aware Optimistic verification protocol for floating-point neural networks that accepts outputs within principled operator-level acceptance regions rather than requiring bitwise equality. TAO combines two complementary error models: (i) sound per-operator IEEE-754 worst-case bounds and (ii) tight empirical percentile profiles calibrated across hardware types. Discrepancies are resolved via a Merkle-anchored, threshold-guided interactive dispute game that recursively partitions the traced computation graph until one operator remains; at the leaf, adjudication reduces to either a lightweight theoretical-bound check or a small honest-majority vote against empirical thresholds. Unchallenged results finalize after a challenge window, without requiring trusted hardware or deterministic kernels. Jianzhu Yao, Hongxu Su, Taobo Liao, Zerui Cheng, Huan Zhang 0001, Xuechao Wang, Pramod Viswanath |
EuroSys | 1 |
| 2025 | LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?abstractRecent reports claim that large language models (LLMs) now outperform elite humans in competitive programming. Drawing on knowledge from a group of medalists in international algorithmic contests, we revisit this claim, examining how LLMs differ from human experts and where limitations still remain. We introduce LiveCodeBench Pro, a benchmark composed of problems from Codeforces, ICPC, and IOI that are continuously updated to reduce the likelihood of data contamination. A team of Olympiad medalists annotates every problem for algorithmic categories and conducts a line-by-line analysis of failed model-generated submissions. Using this new data and benchmark, we find that frontier models still have significant limitations: without external tools, the best model achieves only 53\% pass@1 on medium-difficulty problems and 0\% on hard problems, domains where expert humans still excel. We also find that LLMs succeed at implementation-heavy problems but struggle with nuanced algorithmic reasoning and complex case analysis, often generating confidently incorrect justifications. High performance appears largely driven by implementation precision and tool augmentation, not superior reasoning. LiveCodeBench Pro thus highlights the significant gap to human grandmaster levels, while offering fine-grained diagnostics to steer future improvements in code-centric LLM reasoning. Zihan Zheng, Zerui Cheng, Shang Zhou, Hansen He, Dongruixuan Li, Stanley Wei, Hangyi Hao, Jianzhu Yao, Peiyao Sheng, Zixuan Wang 0029, Wenhao Chai, Aleksandra Korolova, Peter Henderson 0002, Sanjeev Arora, Pramod Viswanath, Jingbo Shang, Saining Xie |
NeurIPS | 10 |