Demo Data from VLDB 2026 / reviewers in the wild / expert
Pierre Brunelle
dblp:243/2466
· DBLP profile ↗
1ranked-venue papers
0as first author
0since 2021 · last 2019
—ORCID · none
Domains — the database's venue-derived domains; a paper can count in several
Databases, data management, data science and information retrieval · 1
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Databases, data mining, and information retrieval
1 paper |
Data integration and cleaning · 50% Data stream processing · 50% |
Topics — the 1 heaviest of 2, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Data stream processing
incremental validation |
0.4 | 1 | 2019 | Unit Testing Data with Deequ · SIGMOD Conference 2019 |
Methods — techniques the papers use, named apart from their topics
declarative constraint API · 0.4anomaly detection on quality metrics · 0.4
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2019 | Unit Testing Data with DeequabstractModern companies and institutions rely on data to guide every single decision. Missing or incorrect information seriously compromises any decision process. We demonstrate "Deequ", an Apache Spark-based library for automating the verification of data quality at scale. This library provides a declarative API, which combines common quality constraints with user-defined validation code, and thereby enables "unit tests for data". Deequ is available as open source, meets the requirements of production use cases at Amazon, and scales to datasets with billions of records if the constraints to evaluate are chosen carefully. Our demonstration walks attendees through a fictitious business use case of validating daily product reviews from a public dataset, and is executed in a proprietary interactive notebook environment. We show attendees how to define data unit tests from automatically suggested constraints and how to create customized tests. Additionally, we demonstrate how to apply Deequ to validate incrementally growing datasets, and give examples of how to configure anomaly detection algorithms on time series of data quality metrics to further automate the data validation. Sebastian Schelter, Felix Bießmann, Dustin Lange, Tammo Rukat, Philipp Schmidt 0002, Stephan Seufert, Pierre Brunelle, Andrey Taptunov |
SIGMOD Conference | 7 |