Demo Data from EDBT 2026 / reviewers in the wild / expert
Tianjiao Lang
dblp:242/4304
· DBLP profile ↗
2ranked-venue papers
0as first author
1since 2021 · last 2024
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 since 2021Applied, interdisciplinary, general and emerging computing · 1
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
1 paper |
Language models and text generation · 100% | |
| Interdisciplinary, comprehensive, and emerging computing
1 paper |
Computing education · 100% |
Topics — the 2 heaviest of 2, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Natural language and speech › Language models and text generation
large language model evaluation |
0.8 | 1 | 2024 | IDGen: Item Discrimination Induced Prompt Generation for LLM Evaluation · NeurIPS 2024 |
Computing education
educational assessment |
0.2 | 1 | 2024 | IDGen: Item Discrimination Induced Prompt Generation for LLM Evaluation · NeurIPS 2024 |
Methods — techniques the papers use, named apart from their topics
self-correct mechanism · 1.5prompt discrimination prediction · 1.5difficulty score prediction · 1.5
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2024 | IDGen: Item Discrimination Induced Prompt Generation for LLM EvaluationabstractAs Large Language Models (LLMs) become more capable of handling increasingly complex tasks, the evaluation set must keep pace with these advancements to ensure it remains sufficiently discriminative. Item Discrimination (ID) theory, which is widely used in educational assessment, measures the ability of individual test items to differentiate between high and low performers. Inspired by this theory, we propose an ID-induced prompt synthesis framework for evaluating LLMs so that the evaluation set continually updates and refines according to model abilities.
Our data synthesis framework prioritizes both breadth and specificity. It can generate prompts that comprehensively evaluate the capabilities of LLMs while revealing meaningful performance differences between models, allowing for effective discrimination of their relative strengths and weaknesses across various tasks and domains.
To produce high-quality data, we incorporate a self-correct mechanism into our generalization framework and develop two models to predict prompt discrimination and difficulty score to facilitate our data synthesis framework, contributing valuable tools to evaluation data synthesis research. We apply our generated data to evaluate five SOTA models. Our data achieves an average score of 51.92, accompanied by a variance of 10.06. By contrast, previous works (i.e., SELF-INSTRUCT and WizardLM) obtain an average score exceeding 67, with a variance below 3.2.
The results demonstrate that the data generated by our framework is more challenging and discriminative compared to previous works.
We will release a dataset of over 3,000 carefully crafted prompts to facilitate evaluation research of LLMs. Fan Lin, Shuyi Xie, Yong Dai 0001, Wenlin Yao, Tianjiao Lang, Yu Zhang 0004 |
NeurIPS | 5 |
| 2019 | Migratory Insect Multifrequency Radar Cross Sections for Morphological Parameter EstimationabstractInsect migration provides major ecosystem services, and sometimes, migratory pests cause serious crop damage and yield loss. Species identification is critically important in studies of insect migration, for both entomologists and pest managers. Radar is an effective means of detecting insect migrants. Current entomological radars usually operate at X-band, and signal amplitude information is used to estimate body mass and wing-beat frequency, which can then be used to categorize migratory insects into broad taxon classes. To improve the identification performance, this paper presents a novel radar method of measuring insect mass and body length. The multifrequency radar cross sections (RCS) of insects at X-band and Ku-/K-band are fully investigated, and the comprehensive relationship between RCS and insect morphological parameters provides an improvement in the estimation of insect mass. More importantly, estimations of body length can also be realized with an accuracy of 84% based on experimental data acquired by a vector network analyzer in a microwave anechoic chamber. If multifrequency RCS measurements can be obtained by radar in the future, then highly accurate estimations of insect mass and body length will be possible, although it is currently still a challenge to build a radar capable of making the required measurements over such a wide frequency range. Rui Wang 0018, Cheng Hu 0001, Teng Long 0001, Shaoyang Kong, Tianjiao Lang, Philip J. L. Gould, Jason Lim, Kongming Wu |
IEEE Trans. Geosci. Remote. Sens. | 6 |