Human-Centered Exploration of Table Unionability

vldb26-2490 · Experiment, Analysis & Benchmark (EA&B) · Nina Klimenkova, Roee Shraga, Sreeram Marimuthu
Abstract

Table union search (TUS) identifies tables that can be meaningfully combined by row-wise union with a given query table and is a core operation in data discovery over data lakes. Yet what it means for two tables to be unionable is inherently ambiguous: domain experts disagree even on seemingly simple cases, and existing benchmarks collapse this disagreement into binary labels, omitting the behavioral context behind human decisions. We take a human-centered view of table unionability and study how humans, traditional TUS methods, and large language models (LLMs) interact on this task. We introduce TUNE (Table UNionability with human Evaluation), a benchmark of 464 expert judgments over 26 table pairs that records binary decisions, confidence scores, decision times, interaction traces, textual explanations, and post-survey reflections. Using TUNE, we (i) characterize human performance, overconfidence, and metacognitive quality (calibration and resolution); (ii) benchmark state-of-the-art TUS methods (Starmie, SANTOS, D3L), revealing complementary strengths and systematic misalignment with expert judgments; and (iii) evaluate four experimental scenarios that combine human behavioral signals and TUS features using classical ML models and LLMs. Our best configuration reaches 84% accuracy, improving over both human majority vote and the strongest standalone TUS method, while LLMs act as useful second opinions but are sensitive to conflicting signals. Overall, our results suggest that unionability labels reflect a structured yet imperfect human decision process and that hybrid human–model pipelines provide more reliable and interpretable unionability assessments.

Assigned reviewers

No reviewers assigned yet.

Candidates from the panel ranked by taxonomy affinity

#ReviewerMatchLoadWhy