Recently, red teaming, with roots in security, has become a key evaluative approach to ensure the safety and reliability of Generative Artificial Intelligence. However, most existing work emphasizes technical benchmarks and attack success rates, leaving the socio-technical practices of how red teaming datasets are defined, created, and evaluated under-examined. Drawing on 22 interviews with practitioners who design and evaluate red teaming datasets, we examine the data practices and standards that underpin this work. Because adversarial datasets determine the scope and accuracy of model evaluations, they are critical artifacts for assessing potential harms from large language models. Our contributions are first, empirical evidence of practitioners conceptualizing red teaming and developing and evaluating red teaming datasets. Second, we reflect on how practitioners’ conceptualization of risk leads to overlooking the context, interaction type, and user specificity. We conclude with three opportunities for HCI researchers to expand the conceptualization and data practices for red-teaming.

Generative AI is rapidly reshaping young people’s digital experiences, from providing emotional support to introducing new dimensions of risks. Yet, existing safety frameworks are not equipped to handle the unique risks posed by GenAI. To investigate how youth safety is being addressed in this new landscape, we conducted a systematic review of (N=30) GenAI-youth studies from 2014-2025. We found that GenAI-youth-related research was primarily led by AI experts with minimal involvement from youth development experts or young people themselves. Safety was typically framed as a technical system feature, optimized through filters, benchmarks, or guardrails, rather than a relational, contextual, and developmentally grounded concern. We call on the HCI community to re-evaluate its approach to participation in AI. We must move beyond reactive, system-driven GenAI approaches to youth safety towards a more holistic, proactive model where multistakeholder inclusion is a core aspect throughout the AI-lifecycle, leading to safer and equitable systems.

Dataset documentation is widely recognized as essential for the responsible development of automated systems. Despite growing efforts to support documentation through different kinds of artifacts, little is known about the motivations shaping documentation tool design or the factors hindering their adoption. We present a systematic review supported by mixed-methods analysis of 59 dataset documentation publications to examine the motivations behind building documentation tools, how authors conceptualize documentation practices, and how these tools connect to existing systems, regulations, and cultural norms. Our analysis shows four persistent patterns in dataset documentation conceptualization that potentially impede adoption and standardization: unclear operationalizations of documentation’s value, decontextualized designs, unaddressed labor demands, and a tendency to treat integration as future work. Building on these findings, we propose a shift in Responsible AI tool design toward institutional rather than individual solutions, and outline actions the HCI community can take to enable sustainable documentation practices.

HCI and CSCW research that uses social media data to make inferences about individuals and communities has proliferated in the last decade. Previous studies have elaborated on methodological concerns and challenges and examined the assumptions and values underlying knowledge production through quantification and data. We expand this line of research by making visible and explicit the conventions and practices that establish, sustain, and reinforce current discourses in social media research. We conducted a Critical Discourse Analysis on 84 research papers published between 2010 and 2023 in CHI, CSCW, and GROUP that combine social media data and computational methods. Our findings show that plenty of this work legitimizes social media data as a valid source of information by centering its public availability, unobtrusiveness, and volume. Furthermore, to justify computational techniques, these papers prioritize computational expediency over data and method appropriateness. We argue that these embedded strategies may result in a methodological and epistemological distance between researchers and the studied communities, impacting problem framing, data collection, and finding application. With this work, we join the voices that have advocated for increased reflexivity in HCI and CSCW communities to scrutinize knowledge production and the role of researchers as knowledge producers.

Youth online safety research in HCI has historically centered on perspectives from the Global North, often overlooking the unique particularities and cultural contexts of regions in the Global South. This paper presents a systematic review of 66 youth online safety studies published between 2014 and 2024, specifically focusing on regions in the Global South. Our findings reveal a concentrated research focus in Asian countries and predominance of quantitative methods. We also found limited research on marginalized youth populations and a primary focus on risks related to cyberbullying. Our analysis underscores the critical role of cultural factors in shaping online safety, highlighting the need for educational approaches that integrate social dynamics and awareness. We propose methodological recommendations and a future research agenda that encourages the adoption of situated, culturally sensitive methodologies and youth-centered approaches to researching youth online safety regions in the Global South. This paper advocates for greater inclusivity in youth online safety research, emphasizing the importance of addressing varied sociocultural contexts to better understand and meet the online safety needs of youth in the Global South.

In this paper, we describe the design and evaluation of the toolkit Bitacora, addressed to practitioners working in non-profit organizations interested in integrating Twitter data into their work. The toolkit responds to the call to maintain the locality of data by promoting a qualitative and contextualized approach to analyzing Twitter data. We assessed the toolkit’s effectiveness in guiding practitioners to search, collect, and be critical when analyzing data from Twitter. We evaluated the toolkit with ten practitioners from three non-profit organizations of different aims and sizes in Mexico. The assessment surfaced tensions between the assumptions embedded in the toolkit’s design and practitioners’ expectations, needs, and backgrounds. We show that practitioners navigated these tensions in some cases by developing strategies and, in others, questioning the appropriateness of using Twitter data to inform their work. We conclude with recommendations for researchers who developed tools for non-profit organizations to inform humanitarian action.

This paper examines the practices involved in mobilizing social media data from their site of production to the institutional context of non-profit organizations. We report on nine months of fieldwork with a transnational and intergovernmental organization using social media data to understand the role of grassroots initiatives in Mexico, in the unique context of the COVID-19 pandemic. We show how different stakeholders negotiate the definition of problems to be addressed with social media data, the collective creation of ground-truth, and the limitations involved in the process of extracting value from data. The meanings of social media data are not defined in advance; instead, they are contingent on the practices and needs of the organization that seeks to extract insights from the analysis. We conclude with a list of reflections and questions for researchers who mediate in the mobilization of social media data into non-profit organizations to inform humanitarian action.

By virtue of being underrepresented in computing, Hispanic women completing a Ph.D. face additional challenges starting their new careers. Completing the dissertation, applying for jobs, interviewing, and negotiating job offers are made more difficult by the lack of a community of mentors that fully understands their intersectional identities. This panel presents the challenges faced and the accomplishments of three recent Hispanic women Ph.D. recipients as they navigated the last stages of the dissertation, applied for jobs, considered job offers, and started their professional careers in computing.

The opacity of machine learning data is a significant threat to ethical data work and intelligible systems. Previous research has addressed this issue by proposing standardized checklists to document datasets. This paper expands that field of inquiry by proposing a shift of perspective: from documenting datasets towards documenting data production. We draw on participatory design and collaborate with data workers at two companies located in Bulgaria and Argentina, where the collection and annotation of data for machine learning are outsourced. Our investigation comprises 2.5 years of research, including 33 semi-structured interviews, five co-design workshops, the development of prototypes, and several feedback instances with participants. We identify key challenges and requirements related to the integration of documentation practices in real-world data production scenarios. Our findings comprise important design considerations and highlight the value of designing data documentation based on the needs of data workers. We argue that a view of documentation as a boundary object, i.e., an object that can be used differently across organizations and teams but holds enough immutable content to maintain integrity, can be useful when designing documentation to retrieve heterogeneous, often distributed, contexts of data production.

Social media platforms offer a rich repository of crowdsourced information that has the potential to monitor human rights violations. The challenge is to quantify, interpret, and situate such unstructured data streams in the broader context, which remains under-investigated in existing CSCW research. Addressing these challenges demands computational solutions to extract large volumes of data in conjunction with human intervention to transition the data streams into the offline context to render them usable and actionable. Following an iterative human-in-the-loop computational approach, we explore whether citizen reports of abductions concentrated on Facebook groups can be useful to complete official records on the ongoing crisis of disappearances in Mexico. We conceptualize three key practices of the process of transitioning the data from online to offline, followed by seven qualitative characteristics of the data streams that contribute to each stage of the process. Our research contributes with an initial understanding of the challenges and opportunities of migrating the local knowledge from online communities to be used as evidence by organizations seeking to address institutional failures.

We examine the local data practices of citizens in Mexico who use Facebook sites as a platform to report crimes and share safety-related information. We conducted 14 interviews with a variety of participants who collaborate as administrators and contributors of these online communities. The communities we examined have two central components: the citizens who crowd-source data about instances of crime in different neighborhoods in and around Mexico City, and the administrators of the Facebook sites who use the crowd-sourced data to intervene and collaborate with other stakeholders. From our interviews, we identify the community, data, and action practices used by group administrators to collect, curate, and publish information about public safety that would otherwise go unreported. The combination of these practices improves the reputation of the groups on Facebook, increases trust, and encourages sustained participation from citizens. These practices also legitimize data gathered by group members as an important grassroots tool for responding to issues of public safety that would otherwise not be reported or acted upon. Our findings contribute a growing body of work that aims to understand how social media enable political action in contexts where people are not being served by existing institutions.

Nearly 40 million workers in the USA, a third of the working population, are low-wage, meaning they make less than $11.65 per hour. These workers face the pervasive and detrimental challenge of wage violations, also known as wage theft, which is any illegal activity by an employer that denies benefits or wages to employees. We interviewed 24 low-wage workers who experienced wage theft and sought justice about their work practices, challenges, and information technology usage. Based on these interviews, we identify three key sociotechnical practices these workers engaged in to address their wage theft: 1) identifying wage and payment discrepancies; 2) tracking and documenting work; and 3) pursuing wage claims. Seeking to leverage HCI research to interrupt uneven social, economic, and information relations in the low-wage workplace, we ultimately reflect on the possibility and limits of several key design recommendations.

In this paper, we examine how activist organizations, focused on human rights violations (HRVs) in Mexico, obtain and translate data to produce actionable insight for social change. Through interviews with 15 participants working in think tanks, human rights centers, non-governmental organizations, and nonprofit organizations, we identified two key data challenges that impact their work: absent and conflicting data. We then describe how these nonprofits try to understand these issues by building alliances to address specific, detrimental knowledge and data gaps. Next, we articulate how these activists use data to work towards social change by informing citizens, requesting action, and building capacity. Lastly, we propose recommendations on how to design for HRVs-focused data practices, focusing on issues related to addressing technology and infrastructure constraints, designing for safety, and supporting community data collection and dissemination.