Demo Data from SIGIR 2026 / reviewers in the wild / expert
Debjani Saha
dblp:256/1114
· DBLP profile ↗
3ranked-venue papers
3as first author
0since 2021 · last 2020
0000-0002-8190-0777ORCID · corroborated
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 2 · 2 first-authorSecurity and privacy · 1 · 1 first-authorHuman-computer interaction and ubiquitous computing · 1 · 1 first-author
Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.
| Artificial intelligence
1 paper |
Trustworthy machine learning · 100% | |
| Human-computer interaction and pervasive computing
1 paper |
Human-AI interaction · 100% |
Topics — the 3 heaviest of 3, each with the papers that count most for it
| Topic | Weight | Papers | Last | Evidence papers |
|---|---|---|---|---|
Machine learning › Trustworthy machine learning
fairness |
0.4 | 1 | 2020 | Measuring Non-Expert Comprehension of Machine Learning Fairness Metrics · ICML 2020 |
Machine learning › Trustworthy machine learning › fairness
fairness evaluation |
0.4 | 1 | 2020 | Measuring Non-Expert Comprehension of Machine Learning Fairness Metrics · ICML 2020 |
Human-AI interaction
user comprehension |
0.4 | 1 | 2020 | Measuring Non-Expert Comprehension of Machine Learning Fairness Metrics · ICML 2020 |
Methods — techniques the papers use, named apart from their topics
online survey · 0.9
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2020 | Human Comprehension of Fairness in Machine LearningabstractBias in machine learning has manifested injustice in several areas, with notable examples including gender bias in job-related ads [4], racial bias in evaluating names on resumes [3], and racial bias in predicting criminal recidivism [1]. In response, research into algorithmic fairness has grown in both importance and volume over the past few years. Different metrics and approaches to algorithmic fairness have been proposed, many of which are based on prior legal and philosophical concepts [2]. The rapid expansion of this field makes it difficult for professionals to keep up, let alone the general public. Furthermore, misinformation about notions of fairness can have significant legal implications. Debjani Saha, Candice Schumann, Duncan C. McElfresh, John Dickerson 0001, Michelle L. Mazurek, Michael Carl Tschantz |
AIES | 1 |
| 2020 | User Attitudes On Direct-to-Consumer Genetic TestingabstractAdvances in biotechnology now allow users to obtain their genetic information, including ancestry and predisposition to various diseases and health issues, with relative ease. With these new commercial services come a host of privacy concerns with respect to data sharing and access. User data is being sold to third parties, including pharmaceutical and biotechnology companies, and may be accessed by law enforcement in accordance with proper legal procedures. Moreover, many users of these services go on to deposit the data they obtain into online, public repositories that are fully accessible to anyone with an internet connection. The full extent of the risks they face may not be apparent to users. This paper reports on a semistructured interview study ($n=24$) examining user concerns regarding these tests, what information they believe they are revealing, and what they think companies are doing with their data. We find that users are concerned with privacy, and understand at a basic level the nature of the data they are revealing. However, their privacy concerns are often insufficient to deter them from taking such a test, and many have difficulty grasping some of the implications of sharing their genetic information with commercial entities. Debjani Saha, Anna Chan, Brook Stacy, Kiran Javkar, Sushant Patkar, Michelle L. Mazurek |
EuroS&P | 1 |
| 2020 | Measuring Non-Expert Comprehension of Machine Learning Fairness MetricsabstractBias in machine learning has manifested injustice in several areas, such as medicine, hiring, and criminal justice. In response, computer scientists have developed myriad definitions of fairness to correct this bias in fielded algorithms. While some definitions are based on established legal and ethical norms, others are largely mathematical. It is unclear whether the general public agrees with these fairness definitions, and perhaps more importantly, whether they understand these definitions. We take initial steps toward bridging this gap between ML researchers and the public, by addressing the question: does a lay audience understand a basic definition of ML fairness? We develop a metric to measure comprehension of three such definitions–demographic parity, equal opportunity, and equalized odds. We evaluate this metric using an online survey, and investigate the relationship between comprehension and sentiment, demographics, and the definition itself. Debjani Saha, Candice Schumann, Duncan C. McElfresh, John Dickerson 0001, Michelle L. Mazurek, Michael Carl Tschantz |
ICML | 1 |