Demo Data from EDBT 2026 / reviewers in the wild / expert
Jingting Zheng
dblp:439/2695
· DBLP profile ↗
1ranked-venue papers
1as first author
1since 2021 · last 2026
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 1 · 1 first-author · 1 since 2021
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
1 paper |
Language models and text generation · 100% |
Topics — the 3 heaviest of 3, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Natural language and speech › Language models and text generation
alignment |
1.0 | 1 | 2026 | Beyond Value Benchmarks: Measuring Value-Structure Alignment in Large Language Models via Symmetric Q-Sorts · ACL (1) 2026 |
Natural language and speech › Language models and text generation
large language model evaluation |
1.0 | 1 | 2026 | Beyond Value Benchmarks: Measuring Value-Structure Alignment in Large Language Models via Symmetric Q-Sorts · ACL (1) 2026 |
Natural language and speech › Language models and text generation › evaluation of language models › reasoning evaluation
moral reasoning evaluation |
1.0 | 1 | 2026 | Beyond Value Benchmarks: Measuring Value-Structure Alignment in Large Language Models via Symmetric Q-Sorts · ACL (1) 2026 |
Methods — techniques the papers use, named apart from their topics
q methodology · 1.0procrustes analysis · 1.0
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | Beyond Value Benchmarks: Measuring Value-Structure Alignment in Large Language Models via Symmetric Q-SortsabstractLarge Language Models (LLMs) are increasingly deployed in contexts requiring complex moral reasoning and value trade-offs.However, existing evaluations typically rely on item-level behavioral metrics, which fail to capture how models structurally prioritize competing values as a cohesive system.To address this, we propose a symmetric human-LLM evaluation framework, grounded in Q methodology, to measure value-structure alignment.Under our protocol, humans and models sort an identical 140-item moral statement set into a shared nine-column forced distribution; for LLMs, we elicit strict rankings and deterministically map them to Q-sort buckets.Using a human reference sample (N = 35), we establish a stable three-factor reference geometry specific to this instrument and sample.We evaluate 12 LLMs across four model families via 240 replicated Q-sorts at two temperature settings, quantifying structural alignment via Procrustes similarity (ϕ) and RSA-based Spearman correlation (ρ).Our results reveal significant crossfamily heterogeneity, model-specific sensitivity to generation stochasticity and localized misalignment, which demonstrate that favorable global scores can obscure underlying regional distortions.While rank-and bucket-based analyses remain highly consistent, prompt phrasing introduces notable variance.Ultimately, assessing value-structure alignment provides a crucial structural complement to traditional itemwise moral benchmarks. Jingting Zheng, Yuqi Ren, Linhao Yu, Yongqi Leng, Deyi Xiong |
ACL (1) | 1 |