Demo Data from EDBT 2026 / reviewers in the wild / expert
Rongchang Song
dblp:116/1053
· DBLP profile ↗
1ranked-venue papers
0as first author
1since 2021 · last 2026
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 since 2021Graphics, computer vision, multimedia, augmented reality and games · 1 · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Computer architecture, parallel and distributed computing, and storage systems
1 paper |
Electronic design automation · 100% | |
| Software engineering, system software, and programming languages
1 paper |
Program synthesis and code generation · 100% |
Topics — the 3 heaviest of 3, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Program synthesis and code generation
code generation with language models |
1.0 | 1 | 2026 | FIXME: Towards End-to-End Benchmarking of LLM-Aided Design Verification · AAAI 2026 |
Electronic design automation › hardware verification and test
functional verification |
1.0 | 1 | 2026 | FIXME: Towards End-to-End Benchmarking of LLM-Aided Design Verification · AAAI 2026 |
Electronic design automation › hardware verification and test
hardware verification |
1.0 | 1 | 2026 | FIXME: Towards End-to-End Benchmarking of LLM-Aided Design Verification · AAAI 2026 |
Methods — techniques the papers use, named apart from their topics
large language model · 2.0benchmarking · 2.0
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | FIXME: Towards End-to-End Benchmarking of LLM-Aided Design VerificationabstractDespite the transformative potential of Large Language Models (LLMs) in hardware design, a comprehensive evaluation of their capabilities in design verification remains underexplored. Current efforts predominantly focus on RTL generation and basic debugging, overlooking the critical domain of functional verification, which is the primary bottleneck in modern design methodologies due to the rapid escalation of hardware complexity. We present FIXME, the first end-to-end, multi-model, and open-source evaluation framework for assessing LLM performance in hardware functional verification (FV) to address this crucial gap. FIXME introduces a structured three-level difficulty hierarchy spanning six verification sub-domains and 180 diverse tasks, enabling in-depth analysis across the design lifecycle. Leveraging a collaborative AI-human approach, we construct a high-quality dataset using 100% silicon-proven designs, ensuring comprehensive coverage of real-world challenges. Furthermore, we enhance the functional coverage by 45.57% through expert-guided optimization. By rigorously evaluating state-of-the-art LLMs such as GPT-4, Claude3, and LlaMA3, we identify key areas for improvement and outline promising research directions to unlock the full potential of LLM-driven automation in hardware design verification. The benchmark is available at https://github.com/ChatDesignVerification/FIXME. Gwok-Waa Wan, Sam-Zaak Wong, Shengchu Su, Chenxu Niu 0001, Ning Wang 0071, Xinlai Wan, Qixiang Chen, Mengnv Xing, Jianmin Ye, Rongchang Song, Qiang Xu 0001, Nan Guan, Zhe Jiang 0004, Xi Wang 0009, Yong Chen 0001, Jun Yang 0006 |
AAAI | 12 |