Nick Merrill

dblp:137/3852 · DBLP profile ↗
← Back
19ranked-venue papers
7as first author
6since 2021 · last 2026
0000-0003-3669-1387ORCID · corroborated

Domains — the database's venue-derived domains; a paper can count in several

Human-computer interaction and ubiquitous computing · 16 · 5 first-author · 5 since 2021Security and privacy · 2 · 1 first-author · 1 since 2021Applied, interdisciplinary, general and emerging computing · 1 · 1 first-author
YearPublicationVenuePosition
2026 Reflections Towards an Ecology of Internet Connectivity: Three Speculative Scenarios Involving Foot Pedals
abstract
HCI’s dominant assumptions of always-on and relatively ubiquitous internet connectivity often overlook other potential configurations of connectivity, which may embody alternative social values and politics, or promote alternative types of technology practices. Building on research exploring alternate configurations of connectivity, we develop and present three speculative scenarios in a North American context that configure internet connectivity differently than these dominant assumptions. Each scenario features a "foot pedal" that mediates internet connectivity. Through the scenarios, we conceptualize connectivity as a multi-dimensional ecology. The scenarios explore how alternative configurations of connectivity implicate concerns related to dimensions of: social norms and rituals; maintenance, repair, and governance; interests and decision-making beyond individual choice; and broader inequalities and systems of power. These suggest possible alternative ends and goals of internet connectivity. Finally, we offer reflections from our experience developing these scenarios for HCI scholars working with speculative practices.
Richmond Y. Wong, Nick Merrill, Robert Soden
CHI2
2024 The Medium is the Message: How Secure Messaging Apps Leak Sensitive Data to Push Notification Services
abstract
Like most modern software, secure messaging apps rely on thirdparty components to implement important app functionality. Although this practice reduces engineering costs, it also introduces the risk of inadvertent privacy breaches due to misconfiguration errors or incomplete documentation. Our research investigated secure messaging apps' usage of Google's Firebase Cloud Messaging (FCM) service to send push notifications to Android devices. We analyzed 21 popular secure messaging apps from the Google Play Store to determine what personal information these apps leak in the payload of push notifications sent via FCM. Of these apps, 11 leaked metadata, including user identifiers (10 apps), sender or recipient names (7 apps), and phone numbers (2 apps), while 4 apps leaked the actual message content. Furthermore, none of the data we observed being leaked to FCM was specifically disclosed in those apps' privacy disclosures. We also found several apps employing strategies to mitigate this privacy leakage to FCM, with varying levels of success. Of the strategies we identified, none appeared to be common, shared, or well-supported. We argue that this is fundamentally an economics problem: incentives need to be correctly aligned to motivate platforms and SDK providers to make their systems secure and private by default.
Nikita Samarin, Alex Sanchez, Trinity Chung, Akshay Dan Bhavish Juleemun, Conor Gilsenan, Nick Merrill, Joel Reardon, Serge Egelman
Proc. Priv. Enhancing Technol.6
2023 The Entoptic Field Camera as Metaphor-Driven Research-through-Design with AI Technologies
abstract
Artificial intelligence (AI) technologies are widely deployed in smartphone photography; and prompt-based image synthesis models have rapidly become commonplace. In this paper, we describe a Research-through-Design (RtD) project which explores this shift in the means and modes of image production via the creation and use of the Entoptic Field Camera. Entoptic phenomena usually refer to perceptions of floaters or bright blue dots stemming from the physiological interplay of the eye and brain. We use the term entoptic as a metaphor to investigate how the material interplay of data and models in AI technologies shapes human experiences of reality. Through our case study using first-person design and a field study, we offer implications for critical, reflective, more-than-human and ludic design to engage AI technologies; the conceptualisation of an RtD research space which contributes to AI literacy discourses; and outline a research trajectory concerning materiality and design affordances of AI technologies.
Jesse Josua Benjamin, Heidi R. Biggs, Arne Berger, Julija Rukanskaite, Michael Heidt, Nick Merrill, James Pierce 0001, Joseph Lindley
CHI6
2023 Accidentally Evil: On Questionable Values in Smart Home Co-Design
abstract
An ongoing mystery of HCI is how do well-intentioned designers consistently enable products with unintentionally evil consequences. Using “questionable values” as a lens, we retell and analyze four design scenarios for smart homes that were created by participants with an IoT toolkit we designed. The selected design scenarios reveal practices that violate principles of responsible smart home design. Through our analysis we show (1) how participants explore sensor-driven objectification of the home then leverage data for surveillance, nudging, and control over others; (2) how the dominant technosolutionist narratives of efficiency and productivity ground such questionable values; (3) and how the materiality of mass-produced sensors pre-mediates questionable design scenarios. We discuss how to attend to and utilize questionable values in design: Making space for questionable values will empower design researchers to better “look around corners”, anticipating tomorrow's concerns and forestalling the worst of their harms.
Arne Berger, Albrecht Kurze, Andreas Bischof, Jesse Josua Benjamin, Richmond Y. Wong, Nick Merrill
CHI6
2023 Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
abstract
Numerous toolkits have been developed to support ethical AI development. However, toolkits, like all tools, encode assumptions in their design about what work should be done and how. In this paper, we conduct a qualitative analysis of 27 AI ethics toolkits to critically examine how the work of ethics is imagined and how it is supported by these toolkits. Specifically, we examine the discourses toolkits rely on when talking about ethical issues, who they imagine should do the work of ethics, and how they envision the work practices involved in addressing ethics. Among the toolkits, we identify a mismatch between the imagined work of ethics and the support the toolkits provide for doing that work. In particular, we identify a lack of guidance around how to navigate labor, organizational, and institutional power dynamics as they relate to performing ethical work. We use these omissions to chart future work for researchers and designers of AI ethics toolkits.
Richmond Y. Wong, Michael A. Madaio, Nick Merrill
Proc. ACM Hum. Comput. Interact.3
2021 Machine Learning Uncertainty as a Design Material: A Post-Phenomenological Inquiry
abstract
Design research is important for understanding and interrogating how emerging technologies shape human experience. However, design research with Machine Learning (ML) is relatively underdeveloped. Crucially, designers have not found a grasp on ML uncertainty as a design opportunity rather than an obstacle. The technical literature points to data and model uncertainties as two main properties of ML. Through post-phenomenology, we position uncertainty as one defining material attribute of ML processes which mediate human experience. To understand ML uncertainty as a design material, we investigate four design research case studies involving ML. We derive three provocative concepts: thingly uncertainty: ML-driven artefacts have uncertain, variable relations to their environments; pattern leakage: ML uncertainty can lead to patterns shaping the world they are meant to represent; and futures creep: ML technologies texture human relations to time with uncertainty. Finally, we outline design research trajectories and sketch a post-phenomenological approach to human-ML relations.
Jesse Josua Benjamin, Arne Berger, Nick Merrill, James Pierce 0001
CHI3
2020 Security Fictions: Bridging Speculative Design and Computer Security
abstract
This paper begins with an observation: that threat identification is an intrinsically speculative practice. It requires imagining possible futures. Drawing on methods from speculative design, this paper presents an improvisational role-playing game designed to help software developers identify security threats. It deploys this game with seven software developers, who used the game to successfully identify diverse threats in their software. The insights from this deployment motivate future work on both the game itself and on organizational accounts of security. I call on the design research community to continue to apply its methods and perspectives to computer security, locating threat identification itself, like all speculation, as a site of social and political power.
Nick Merrill
Conference on Designing Interactive Systems1
2020 Expanding Modes of Reflection in Design Futuring
abstract
Design futuring approaches, such as speculative design, design fiction and others, seek to (re)envision futures and explore alternatives. As design futuring becomes established in HCI design research, there is an opportunity to expand and develop these approaches. To that end, by reflecting on our own research and examining related work, we contribute five modes of reflection. These modes concern formgiving, temporality, researcher positionality, real-world engagement, and knowledge production. We illustrate the value of each mode through careful analysis of selected design exemplars and provide questions to interrogate the practice of design futuring. Each reflective mode offers productive resources for design practitioners and researchers to articulate their work, generate new directions for their work, and analyze their own and others' work.
Sandjar Kozubaev, Chris Elsden, Noura Howell, Marie Louise Juul Søndergaard, Nick Merrill, Britta Friederike Schulte, Richmond Y. Wong
CHI5
2020 Sensor Illumination: Exploring Design Qualities and Ethical Implications of Smart Cameras and Image/Video Analytics
abstract
Drawing analogies between smart cameras and electric lighting, we highlight and extrapolate design trends towards always-on sensing in intimate contexts, and the functional expansion of smart cameras as general-purpose and multi-functional devices. Employing a research through design (RtD) approach, we extrapolate these trends using speculative scenarios, materialize the scenarios by designing and constructing lighting-inspired smart camera fixtures, and self-experiment with these fixtures to introduce and exacerbate privacy and security issues, and inspire creative workarounds and design opportunities for sensor-level regulation. We synthesize our insights by presenting 8 smart camera sensing design qualities for addressing privacy, security, and related social and ethical issues.
James Pierce 0001, Richmond Y. Wong, Nick Merrill
CHI3
2020 Infrastructural Speculations: Tactics for Designing and Interrogating Lifeworlds
abstract
This paper introduces "infrastructural speculations," an orientation toward speculative design that considers the complex and long-lived relationships of technologies with broader systems, beyond moments of immediate invention and design. As modes of speculation are increasingly used to interrogate questions of broad societal concern, it is pertinent to develop an orientation that foregrounds the "lifeworld" of artifacts-the social, perceptual, and political environment in which they exist. While speculative designs often imply a lifeworld, infrastructural speculations place lifeworlds at the center of design concern, calling attention to the cultural, regulatory, environmental, and repair conditions that enable and surround particular future visions. By articulating connections and affinities between speculative design and infrastructure studies research, we contribute a set of design tactics for producing infrastructural speculations. These tactics help design researchers interrogate the complex and ongoing entanglements among technologies, institutions, practices, and systems of power when gauging the stakes of alternate lifeworlds.
Richmond Y. Wong, Vera D. Khovanskaya, Sarah E. Fox, Nick Merrill, Phoebe Sengers
CHI4
2019 Sensing is Believing: What People Think Biosensors Can Reveal About Thoughts and Feelings
abstract
Biosensors-devices that sense the human body-are increasingly ubiquitous. However, it is unclear how people evaluate the risks associated with their use, in part because it is not well-understood what people believe these sensors can reveal. In this study, participants ranked biosensors by how likely they are to reveal what a person is thinking and feeling. We report quantitative and qualitative results of two survey-based studies, one on Mechanical Turk workers (n=100), and one on participants in a longitudinal self-tracking study (n=100). Our findings imply that, in the absence of information about particular sensing technologies, people rely on existing beliefs about the body to explain what they might reveal. Highlighting mismatches between perceived and actual technical capabilities, we contribute recommendations for designers and users.
Nick Merrill, John C.-I. Chuang, Coye Cheshire
Conference on Designing Interactive Systems1
2018 An Interface without A User: An Exploratory Design Study of Online Privacy Policies and Digital Legalese
abstract
Privacy policies are critical to understanding one's rights on online platforms, yet few users read them. In this pictorial, we approach this as a systemic issue that is part a failure of interaction design. We provided a variety of people with printed packets of privacy policies, aiming to tease out this form's capabilities and limitations as a design interface, to understand people's perception and uses, and to critically imagine pragmatic revisions and creative alternatives to existing privacy policies.
James Pierce 0001, Sarah E. Fox, Nick Merrill, Richmond Y. Wong, Carl F. DiSalvo
Conference on Designing Interactive Systems3
2018 When BCIs have APIs: Design Fictions of Everyday Brain-Computer Interface Adoption
abstract
In this paper, we use design fiction to explore the social implications for adoption of brain-computer interfaces (BCI). We argue that existing speculations about BCIs are incomplete: they discuss fears about radical changes in types of control, at the expense of discussing more traditional types of power that emerge in everyday experience, particularly via labor. We present a design fiction in which a BCI technology creates a new type of menial labor, using workers' unconscious reactions to assist algorithms in performing a sorting task. We describe how such a scenario could unfold through multiple sites of interaction: the design of an API, a programmer's question on StackOverflow, an internal memo from a dating company, and a set of forum posts about laborers' experience using the designed system. Through these fictions, we deepen and expand conversations around what kinds of (everyday) futures BCIs could create.
Richmond Y. Wong, Nick Merrill, John C.-I. Chuang
Conference on Designing Interactive Systems2
2018 From Scanning Brains to Reading Minds: Talking to Engineers about Brain-Computer Interface
abstract
We presented software engineers in the San Francisco Bay Area with a working brain-computer interface (BCI) to surface the narratives and anxieties around these devices among technical practitioners. Despite this group's heterogeneous beliefs about the exact nature of the mind, we find a shared belief that the contents of the mind will someday be "read' or "decoded' by machines. Our findings help illuminate BCI's imagined futures among engineers. We highlight opportunities for researchers to involve themselves preemptively in this nascent space of intimate biosensing devices, suggesting our findings' relevance to long-term futures of privacy and cybersecurity.
Nick Merrill, John C.-I. Chuang
CHI1
2018 Differential Vulnerabilities and a Diversity of Tactics: What Toolkits Teach Us about Cybersecurity
abstract
We investigate cybersecurity toolkits, collections of public facing materials intended to help users achieve security online. Through a qualitative analysis of 41 online toolkits, we present a set of key design dimensions: agentive scale (who is responsible for security), achievability (can security be achieved), and interventional stage (when are security measures taken). Recognizing toolkits as socially and culturally situated, we surface ways in which toolkits construct security as a value and, in so doing, how they construct people as (in)secure users. We center the notion of differential vulnerabilities, an understanding of security that recognizes safety as socially contingent, adversaries as unstable figures, and risk as differentially applied based on markers of relational position (e.g. class, race, religion, gender, geography, experience). We argue that differential vulnerabilities provides a key design concern in future security resources, and a critical concept for security discourses.
James Pierce 0001, Sarah E. Fox, Nick Merrill, Richmond Y. Wong
Proc. ACM Hum. Comput. Interact.3
2017 Trust Your Heart: Assessing Cooperation and Trust with Biosignals in Computer-Mediated Interactions
abstract
This study examines heartrate sharing in the context of a trust-building game. Through quantitative and qualitative analyses, we find that "elevated" (versus "normal") heartrate of an exchange partner is associated with negative mood attributions and reduced cooperation in a social dilemma game. To investigate how specific our findings are to heartrate (as opposed to some other "elevated" signal collected from the body), we replicate our initial experiment with an unfamiliar biosignal, "skin reflectivity". We find that both heartrate and the unfamiliar biosignal are associated with negative mood attributions, but we observe a decrease in cooperative behavior only with elevated heartrate. Qualitative results indicate that individuals may learn an association between our unfamiliar biosignal and the cooperative, trusting behavior of their partner. Our findings highlight the role prior beliefs can play in shaping interpretations of a biosignal, while suggesting that designers can, perhaps inadvertently, train users to associate signals with social meanings. We discuss implications for how wearable sensors can mediate social interactions.
Nick Merrill, Coye Cheshire
CSCW1
2017 Is the Future of Authenticity All In Our Heads?: Moving Passthoughts From the Lab to the World
abstract
Passthoughts, in which a user thinks a secret thought to log in to services or devices, provides two factors of authentication (knowledge and inherence) in a single step. Since its proposal in 2005, passthoughts enjoyed a number of successful empirical studies. In this paper, we renew the promise of passthoughts authentication, outlining the main challenges that passthoughts must overcome in order to move from the lab to the real world. We propose two studies, which seek different angles at the fundamental questions we pose. Further, we propose it as a fruitful case study for thinking about what authentication can, and should, be expected to do, as it pushes up against questions of what sorts of "selves" authentication systems must be tasked with recognizing. Through this discussion, we raise novel possibilities for authentication broadly, such as "organic passwords" that change naturally over time, or systems that reject users who are not acting quite "like themselves."
Nick Merrill, Max T. Curran, John C.-I. Chuang
NSPW1
2016 Classifying mental gestures with in-ear EEG
abstract
While brain-computer interfaces (BCI) based on electroencephalography (EEG) have improved dramatically over the past five years, their inconvenient, head-worn form factor has challenged their wider adoption. In this paper, we investigate how EEG signals collected from the ear could be used for “gestural” control of a brain-computer interface (BCI). Specifically, we investigate the efficacy of a support vector classifier (SVC) in distinguishing between mental tasks, or gestures, recorded by a modified, consumer headset. We find that an SVC reaches acceptable BCI accuracy for nine of the subjects in our pool (n=12), and distinguishes at least one pair of gestures better than chance for all subjects. User surveys highlight the need for longer-term research on user attitudes toward in-ear EEG devices, for discreet, non-invasive BCIs.
Nick Merrill, Max T. Curran, Jong-Kai Yang, John C.-I. Chuang
BSN1
2016 Habits of the Heart(rate): Social Interpretation of Biosignals in Two Interaction Contexts
abstract
We investigate interpretations of a biosignal (heartrate) in uncertain social interactions. We describe the quantitative and qualitative results of a randomized vignette experiment in which subjects were asked to make assessments about an acquaintance based on an imagined scenario that included shared heartrate information. We compare the results of this experiment in adversarial and non-adversarial contexts of interaction. We find that elevated heartrate transmits cues about mood in both contexts, but that these cues do not appear to impact assessments of trustworthiness, reliability and dependability. Counter to our expectations, we find that normal (rather than elevated) heartrate leads to negative trust-related assessments, but only in an adversarial context. Our qualitative analysis points to the role of social expectations in shaping contextual interpretations of heartrate, and reveals individual differences in the way interpretations are constructed. We unpack some of the ways that social meanings can arise from biosensor data, and discuss considerations for those designing interactions with wearables.
Nick Merrill, Coye Cheshire
GROUP1