VLDB 2026 Research / reviewers in the wild / expert
Matthew Zhao
dblp:408/2272
· DBLP profile ↗
1ranked-venue papers
0as first author
1since 2021 · last 2025
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Software engineering, system software, and programming languages
1 paper |
Program synthesis and code generation · 50% Requirements engineering and software design · 25% Program verification · 25% |
Topics — the 4 heaviest of 5, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Program synthesis and code generation
code generation with language models |
0.9 | 1 | 2025 | CLEVER: A Curated Benchmark for Formally Verified Code Generation · NeurIPS 2025 |
Program verification
proof assistants |
0.9 | 1 | 2025 | CLEVER: A Curated Benchmark for Formally Verified Code Generation · NeurIPS 2025 |
Requirements engineering and software design › specification
specification generation |
0.9 | 1 | 2025 | CLEVER: A Curated Benchmark for Formally Verified Code Generation · NeurIPS 2025 |
Program synthesis and code generation › formal synthesis
verified code generation |
0.9 | 1 | 2025 | CLEVER: A Curated Benchmark for Formally Verified Code Generation · NeurIPS 2025 |
Methods — techniques the papers use, named apart from their topics
type checking · 0.9large language model · 0.9agentic approaches · 0.9
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | CLEVER: A Curated Benchmark for Formally Verified Code GenerationabstractWe introduce ${\rm C{\small LEVER}}$, a high-quality, manually curated benchmark of 161 problems for end-to-end verified code generation in Lean. Each problem consists of (1) the task of generating a specification that matches a held-out ground-truth specification, and (2) the task of generating a Lean implementation that provably satisfies this specification. Unlike prior benchmarks,${\rm C{\small LEVER}}$ avoids test-case supervision, LLM-generated annotations, and specifications that leak implementation logic or allow vacuous solutions. All outputs are verified post-hoc using Lean's type checker to ensure machine-checkable correctness. We use ${\rm C{\small LEVER}}$ to evaluate several few-shot and agentic approaches based on state-of-the-art language models. These methods all struggle to achieve full verification, establishing it as a challenging frontier benchmark for program synthesis and formal reasoning. Our benchmark can be found on [GitHub](https://github.com/trishullab/clever) as well as [HuggingFace](https://huggingface.co/datasets/amitayusht/clever). All our evaluation code is also available [online](https://github.com/trishullab/clever-prover). Amitayush Thakur, Jasper Lee, George Tsoukalas, Meghana Aparna Sistla, Matthew Zhao, Stefan Zetzsche, Greg Durrett, Yisong Yue, Swarat Chaudhuri |
NeurIPS | 5 |