The rapid expansion of the AI industry relies heavily on the production, verification, and maintenance of data, otherwise known as "data work". Companies outsource and offshore this work through global AI supply chains that operate under exploitative conditions. Drawing on semi-structured interviews with Kenyan data workers across platforms and BPOs, this paper examines how such conditions take shape and persist. We argue that workers are caught within a regime of entrapment, a system of interconnected mechanisms that make it difficult for workers to leave or improve their positions. These mechanisms include the push to invest in the promise of ‘AI’ jobs, the use of precarious contracts to govern workers, the capture of regulatory institutions, and the exploitation of global labor arbitrage. Using complementary lenses of neoliberal governmentality, precarity, and supply chain capitalism, we analyze why labor mobilization in this sector remains uniquely constrained. We conclude by outlining an orientation for research and scholarly practice that can support workers’ organizing efforts and contest the structural conditions sustaining this regime.

Data powering AI is often opaque. Researchers, NGOs, and law and policy leaders have called for greater transparency about how data is used for training, fine-tuning, and evaluation. While data transparency is often championed as crucial, what it concretely enables is largely implicit. Similarly, the concerns developers seem to have about transparency go unstated. This lack of clarity has led some researchers to critique transparency demands as disconnected from the actual benefits—or risks—to specific stakeholders. We analyze documentation from four stakeholder groups to create a taxonomy of the risks and benefits of dataset transparency. Data transparency is perceived as either a risk or a benefit given a stakeholder’s position, rather than wholesale. We also propose data availability and data documentation as two lenses through which to consider transparency. We discuss how best to strategically promote situational data transparency that takes into account the relationship between stakeholder position, transparency modality, and benefits/risks.

Data workers play a key role in the big data industry. Clients hire data workers to collect and annotate data with human identity concepts, like demographic categories or clothing items. Often, such workers are treated as computational-they are expected to quickly and objectively conduct their work, with the goal of having huge, unbiased datasets for training models. Computer vision is especially interested in fair and impartial data due to biases and unethical practices in the field. However, far from impartial, data workers imbue computer vision data with ''biases'' beyond correct versus incorrect answers. Data workers embed their own specific positional perspectives about identity concepts in both collection and annotation processes. Through interviews and ethnographic observations of data workers (freelance and business process outsourcing (BPO) employees), we show how worker positionality influences decisions during data work. We also show the unintended outcomes, like social biases, that occur when positionality is not explicitly attended to in client instructions. We discuss how employing a lens of positionality in data work reveals the gulfs between data worker perspectives and client expectations, which are colored by a web of positional actors beyond isolated data workers. We propose positional (il)legibility as an approach to data work that embraces the reality of positionality in classification practices and addresses the failures of positivist bias mitigation practices.

In a world with shrinking queer spaces, dating apps serve as a useful way to find romantic or sexual partners, make friends, and develop connections in an often fragmented community. However, dating app structures may exclude queer users more than their cisgender heterosexual counterparts. In this paper, we examine how gender and sexuality are formatted as data in dating app profiles and algorithmically curated matchmaking functions. Based on an analysis of ten popular dating apps, we found gender and sexuality were often divided into descriptive labels and functional categories, which systematically marginalize or exclude queer identities from the matching process despite appearing inclusive. We present a framework for identity on dating apps, disentangling identity into a multi-layered construct of data schemas, profiles, algorithms, and platform ideology, thus enabling designers of data-driven systems to identify salient factors for users, even beyond queer identities and dating apps.

Large language models (LLMs) are the new hot trend being rapidly integrated into products and services—often, in chatbots. LLM-powered chatbots are expected to respond to any number of topics, including topics central to gender identity . In light of rising anti-trans discourse, we examined how two popular LLMs responded to real-world English-language questions about trans identity taken from Quora. We employed reflexive analysis that centered our situated knowledges of the trans community. We found that LLMs return pro-trans responses, even when presented with highly transphobic user prompts. While we also found highly transphobic LLM responses, we found that anti-trans sentiment in LLMs was often subtle, requiring a deep positional understanding from diverse trans stakeholders to interpret. Based on these findings, we recommend diverging from current “value-neutral” approaches that validate transphobia by taking an “all sides” approach. We provide considerations for both the evaluation and design of LLMs that center positional expertise.

There has been a great deal of scholarly attention on issues of identity-related bias in machine learning. Much of this attention has focused on data and data workers, workers who do annotation tasks. Yet tech workers—like engineers, data scientists, and researchers—introduce their own “biases” when defining “identity” concepts. More specifically, they instill their own positionalities, the way they understand and are shaped by the world around them. Through interviews with industry tech workers who focus on computer vision, we show how workers embed their own positional perspectives into products and how positional gaps can lead to unforeseen and undesirable outcomes. We discuss how worker positionality is mutually shaped by the contexts in which they are embedded. We provide implications for researchers and practitioners to engage with the positionalities of tech workers, as well as those in contexts outside of development that influence tech workers.

Despite extensive efforts to create fairer machine learning (ML) datasets, there remains a limited understanding of the practical aspects of dataset curation. Drawing from interviews with 30 ML dataset curators, we present a comprehensive taxonomy of the challenges and trade-offs encountered throughout the dataset curation lifecycle. Our findings underscore overarching issues within the broader fairness landscape that impact data curation. We conclude with recommendations aimed at fostering systemic changes to better facilitate fair dataset curation practices.

In this paper, we examine computational approaches for measuring the "fairness" of image tagging systems, finding that they cluster into five distinct categories, each with its own analytic foundation. We also identify a range of normative concerns that are often collapsed under the terms "unfairness," "bias," or even "discrimination" when discussing problematic cases of image tagging. Specifically, we identify four types of representational harms that can be caused by image tagging systems, providing concrete examples of each. We then consider how different computational measurement approaches map to each of these types, demonstrating that there is not a one-to-one mapping. Our findings emphasize that no single measurement approach will be definitive and that it is not possible to infer from the use of a particular measurement approach which type of harm was intended to be measured. Lastly, equipped with this more granular understanding of the types of representational harms that can be caused by image tagging systems, we show that attempts to mitigate some of these types of harms may be in tension with one another.

Computer vision is a "data hungry" field. Researchers and practitioners who work on human-centric computer vision, like facial recognition, emphasize the necessity of vast amounts of data for more robust and accurate models. Humans are seen as a data resource which can be converted into datasets. The necessity of data has led to a proliferation of gathering data from easily available sources, including "public" data from the web. Yet the use of public data has significant ethical implications for the human subjects in datasets. We bridge academic conversations on the ethics of using publicly obtained data with concerns about privacy and agency associated with computer vision applications. Specifically, we examine how practices of dataset construction from public data-not only from websites, but also from public settings and public records-make it extremely difficult for human subjects to trace their images as they are collected, converted into datasets, distributed for use, and, in some cases, retracted. We discuss two interconnected barriers current data practices present to providing an ethics of traceability for human subjects: awareness and control. We conclude with key intervention points for enabling traceability for data subjects. We also offer suggestions for an improved ethics of traceability to enable both awareness and control for individual subjects in dataset curation practices.

The study of eating disorders online has a long tradition within CSCW and HCI scholarship. Research within this body of work highlights the types of content people with eating disorders post as well as the ways in which individuals use online spaces for acceptance, connection, and support. However, despite nearly a decade of research, online eating disorder scholarship in CSCW and HCI rarely accounts for the ways gender shapes online engagement. In this paper, we present empirical results from interviews with 14 trans people with eating disorders. Our findings illustrate how working with gender as an analytic lens allowed us to produce new knowledge about the embodiment of participation in online eating disorder spaces. We show how trans people with eating disorders use online eating disorder content to inform and set goals for their bodies and how, as gender minorities within online eating disorder spaces, trans people occupy marginal positions that make them more susceptible to harms, such as threats to eating disorder validity and gender authenticity. In our discussion, we consider life transitions in the context of gender and eating disorders and address how online eating disorder spaces operate as social transition machinery. We also call attention to the labor associated with online participation as a gender minority within online eating disorder spaces, outlining several design recommendations for supporting the ways trans people with eating disorders use online spaces. CONTENT WARNING: This paper is about the online experiences of trans people with eating disorders. We discuss eating disorders, related content (e.g., thinspiration) and practices (e.g., binge eating, restriction), and gender dysphoria. Please read with caution.

Content creators are instructed to write textual descriptions of visual content to make it accessible; yet existing guidelines lack specifics on how to write about people’s appearance, particularly while remaining mindful of consequences of (mis)representation. In this paper, we report on interviews with screen reader users who were also Black, Indigenous, People of Color, Non-binary, and/or Transgender on their current image description practices and preferences, and experiences negotiating theirs and others’ appearances non-visually. We discuss these perspectives, and the ethics of humans and AI describing appearance characteristics that may convey the race, gender, and disabilities of those photographed. In turn, we share considerations for more carefully describing appearance, and contexts in which such information is perceived salient. Finally, we offer tensions and questions for accessibility research to equitably consider politics and ecosystems in which technologies will embed, such as potential risks of human and AI biases amplifying through image descriptions.

Gender input forms act as gates to accessing information, websites, and services online. Non-binary people regularly have to interact with them, though many do not offer non-binary gender options. This results in non-binary individuals having to either choose an incorrect gender category or refrain from using a site or service—which is occasionally infeasible (e.g., when accessing health services). We tested five different forms through a survey with binary and non-binary participants (n = 350) in three contexts—a digital health form, a social media website, and a dating app. Our results indicate that the majority of participants found binary “male or female” forms exclusive and uncomfortable to fill out across all contexts. We conclude with design considerations for improving gender input forms and consequently their underlying gender model in databases. Our work aims to sensitize designers of (online) gender web forms to the needs and desires of non-binary people.

Data is a crucial component of machine learning. The field is reliant on data to train, validate, and test models. With increased technical capabilities, machine learning research has boomed in both academic and industry settings, and one major focus has been on computer vision. Computer vision is a popular domain of machine learning increasingly pertinent to real-world applications, from facial recognition in policing to object detection for autonomous vehicles. Given computer vision's propensity to shape machine learning research and impact human life, we seek to understand disciplinary practices around dataset documentation - how data is collected, curated, annotated, and packaged into datasets for computer vision researchers and practitioners to use for model tuning and development. Specifically, we examine what dataset documentation communicates about the underlying values of vision data and the larger practices and goals of computer vision as a field. To conduct this study, we collected a corpus of about 500 computer vision datasets, from which we sampled 114 dataset publications across different vision tasks. Through both a structured and thematic content analysis, we document a number of values around accepted data practices, what makes desirable data, and the treatment of humans in the dataset construction process. We discuss how computer vision datasets authors value efficiency at the expense of care; universality at the expense of contextuality; impartiality at the expense of positionality; and model work at the expense of data work. Many of the silenced values we identify sit in opposition with social computing practices. We conclude with suggestions on how to better incorporate silenced values into the dataset creation and curation process.

The proliferation of harmful content on online social media platforms has necessitated empirical understandings of experiences of harm online and the development of practices for harm mitigation. Both understandings of harm and approaches to mitigating that harm, often through content moderation, have implicitly embedded frameworks of prioritization-what forms of harm should be researched, how policy on harmful content should be implemented, and how harmful content should be moderated. To aid efforts of better understanding the variety of online harms, how they relate to one another, and how to prioritize harms relevant to research, policy, and practice, we present a theoretical framework of severity for harmful online content. By employing a grounded theory approach, we developed a framework of severity based on interviews and card-sorting activities conducted with 52 participants over the course of ten months. Through our analysis, we identified four Types of Harm (physical, emotional, relational, and financial) and eight Dimensions along which the severity of harm can be understood (perspectives, intent, agency, experience, scale, urgency, vulnerability, sphere). We describe how our framework can be applied to both research and policy settings towards deeper understandings of specific forms of harm (e.g., harassment) and prioritization frameworks when implementing policies encompassing many forms of harm.

Coming out and being visible online can offer transgender and/or non-binary people benefits not found elsewhere. However, it also can expose them to negative reactions and bad experiences. Through an analysis of 15 semi-structured interviews, we investigate the experiences of transgender and/or non-binary people coming out across social media sites (SMSs). We found that participants employed strategies around disclosure and visibility to limit the consequences of coming out and to access support. Using trans theory on visibility, we discuss how online spaces present metaphorical "doors" to resources, support, and recognition---but can also be "traps" for those that do not meet the expectations of the space. We discuss how visibility empowered participants to create "trapdoors" to new spaces within SMSs where they could create positive outcomes for themselves and their communities. We close with considerations for designers as they create online spaces, and present a broader call to action for the HCI community around designing online spaces.

Race and gender have long sociopolitical histories of classification in technical infrastructures-from the passport to social media. Facial analysis technologies are particularly pertinent to understanding how identity is operationalized in new technical systems. What facial analysis technologies can do is determined by the data available to train and evaluate them with. In this study, we specifically focus on this data by examining how race and gender are defined and annotated in image databases used for facial analysis. We found that the majority of image databases rarely contain underlying source material for how those identities are defined. Further, when they are annotated with race and gender information, database authors rarely describe the process of annotation. Instead, classifications of race and gender are portrayed as insignificant, indisputable, and apolitical. We discuss the limitations of these approaches given the sociohistorical nature of race and gender. We posit that the lack of critical engagement with this nature renders databases opaque and less trustworthy. We conclude by encouraging database authors to address both the histories of classification inherently embedded into race and gender, as well as their positionality in embedding such classifications.

Investigations of facial analysis (FA) technologies-such as facial detection and facial recognition-have been central to discussions about Artificial Intelligence's (AI) impact on human beings. Research on automatic gender recognition, the classification of gender by FA technologies, has raised potential concerns around issues of racial and gender bias. In this study, we augment past work with empirical data by conducting a systematic analysis of how gender classification and gender labeling in computer vision services operate when faced with gender diversity. We sought to understand how gender is concretely conceptualized and encoded into commercial facial analysis and image labeling technologies available today. We then conducted a two-phase study: (1) a system analysis of ten commercial FA and image labeling services and (2) an evaluation of five services using a custom dataset of diverse genders using self-labeled Instagram images. Our analysis highlights how gender is codified into both classifiers and data standards. We found that FA services performed consistently worse on transgender individuals and were universally unable to classify non-binary genders. In contrast, image labeling often presented multiple gendered concepts. We also found that user perceptions about gender performance and identity contradict the way gender performance is encoded into the computer vision infrastructure. We discuss our findings from three perspectives of gender identity (self-identity, gender performativity, and demographic identity) and how these perspectives interact across three layers: the classification infrastructure, the third-party applications that make use of that infrastructure, and the individuals who interact with that software. We employ Bowker and Star's concepts of "torque" and "residuality" to further discuss the social implications of gender classification. We conclude by outlining opportunities for creating more inclusive classification infrastructures and datasets, as well as with implications for policy.

Automatic Gender Recognition (AGR) refers to various computational methods that aim to identify an individual's gender by extracting and analyzing features from images, video, and/or audio. Applications of AGR are increasingly being explored in domains such as security, marketing, and social robotics. However, little is known about stakeholders' perceptions and attitudes towards AGR and how this technology might disproportionately affect vulnerable communities. To begin to address these gaps, we interviewed 13 transgender individuals, including three transgender technology designers, about their perceptions and attitudes towards AGR. We found that transgender individuals have overwhelmingly negative attitudes towards AGR and fundamentally question whether it can accurately recognize such a subjective aspect of their identity. They raised concerns about privacy and potential harms that can result from being incorrectly gendered, or misgendered, by technology. We present a series of recommendations on how to accommodate gender diversity when designing new digital systems.

Transgender individuals in the United States face significant threats to interpersonal safety; however, there has as yet been relatively little research in the HCI and CSCW communities to document transgender individuals' experiences of technology-mediated safety and harm. In this study, we interviewed 12 transgender and non-binary individuals to understand how they find, create, and navigate safe spaces using technology. Managing safety was a universal concern for our transgender participants, and they experienced complex manifestations of harm through technology. We found that harmful experiences for trans users could arise as targeted or incidental affronts, as sourced from outsiders or insiders, and as directed against individuals or entire communities.. Notably, some violations implicated technology design, while others tapped broader social dynamics. Reading our findings through the notions of 'space" and 'place," we unpack challenges and opportunities for building safer futures with transfolk, other vulnerable users, and their allies.

For decades, researchers have investigated and developed technologies that support independent navigation for people who are blind. This has led to systems that primarily aid in detecting routes, landmarks, and building features. However, there has been relatively little inquiry regarding how technologies might support navigation around and in the presence of other people. What visual information, if any, do blind navigators wish they had about people on their path? To address this question, we surveyed 58 blind and low vision individuals and interviewed 10 blind individuals. We discovered our participants were interested in using visual information about others to increase their physical safety. For example, they wanted to know if a passerby was holding a weapon, if a presumed official had a proper uniform or badge, and how to describe visual aspects of a criminal to law enforcement. This paper presents one of the only reports documenting accessibility challenges related to physical safety posed by others, including how future assistive tools can empower individuals with disabilities to more actively increase their sense of safety. We call this emerging area Personal Safety Management and contribute a set of four broad subareas that deserve further exploration by researchers and designers working within the blind and broader disabilities communities.