Our study of 20 knowledge workers revealed a common challenge: the difficulty of synthesizing unstructured information scattered across multiple platforms to make informed decisions. Drawing on their vision of an ideal knowledge synthesis tool, we developed Yodeai, an AI-enabled system, to explore both the opportunities and limitations of AI in knowledge work. Through a user study with 16 product managers, we identified three key requirements for Generative AI in knowledge work: adaptable user control, transparent collaboration mechanisms, and the ability to integrate background knowledge with external information. However, we also found significant limitations, including overreliance on AI, user isolation, and contextual factors outside the AI's reach. As AI tools become increasingly prevalent in professional settings, we propose design principles that emphasize adaptability to diverse workflows, accountability in personal and collaborative contexts, and context-aware interoperability to guide the development of human-centered AI systems for product managers and knowledge workers.

Public records requests are a central mechanism for government transparency. In practice, they are slow, complex processes that require analyzing large amounts of messy, unstructured data. In this paper, we introduce RequestAtlas, a system that helps investigative journalists review large quantities of unstructured data that result from submitting many public records requests. RequestAtlas was developed through a year-long participatory design collaboration with the California Reporting Project (CRP), a journalistic collective researching police use of force and police misconduct in California. RequestAtlas helps journalists evaluate the results of public records requests for completeness and negotiate with agencies for additional information. RequestAtlas has had significant real-world impact. It has been deployed for more than a year to identify missing data in response to public records requests and to facilitate negotiation with public records request officers. Through the process of designing and observing the use of RequestAtlas, we explore the technical challenges associated with the public records request process and the design needs of investigative journalists more generally. We argue that public records requests represent an instance of an adversarial technical relationship in which two entities engage in a prolonged, iterative, often adversarial exchange of information. Technologists can support information-gathering efforts within these adversarial technical relationships by building flexible local solutions that help both entities account for the state of the ongoing information exchange. Additionally, we offer insights on ways to design applications that can assist investigative journalists in the inevitably significant data cleaning phase of processing large documents while supporting journalistic norms of verification and human review. Finally, we reflect on the ways that this participatory design process, despite its success, lays bare some of the limitations inherent in the public records request process and in the ''request and respond'' model of transparency more generally.

Online interpersonal harm, such as cyberbullying and sexual harassment, remains a pervasive issue on social media platforms. Traditional approaches, primarily content moderation, often overlook survivors' needs and agency. We introduce SnuggleSense, a system that empowers survivors through structured sensemaking. Inspired by restorative justice practices, SnuggleSense guides survivors through reflective questions, offers personalized recommendations from similar survivors, and visualizes plans using interactive sticky notes. A controlled experiment demonstrates that SnuggleSense significantly enhances sensemaking compared to an unstructured process of making sense of the harm. We argue that SnuggleSense fosters community awareness, cultivates a supportive survivor network, and promotes a restorative justice-oriented approach toward restoration and healing. We also discuss design insights, such as tailoring informational support and providing guidance while preserving survivors' agency.

Accountable use of AI systems in high-stakes settings relies on making systems contestable. In this paper we study efforts to contest AI systems in practice by studying how public defenders scrutinize AI in court. We present findings from interviews with 17 people in the U.S. public defense community to understand their perceptions of and experiences scrutinizing computational forensic software (CFS) — automated decision systems that the government uses to convict and incarcerate, such as facial recognition, gunshot detection, and probabilistic genotyping tools. We find that our participants faced challenges assessing and contesting CFS reliability due to difficulties (a) navigating how CFS is developed and used, (b) overcoming judges and jurors’ non-critical perceptions of CFS, and (c) gathering CFS expertise. To conclude, we provide recommendations that center the technical, social, and institutional context to better position interventions such as performance evaluations to support contestability in practice.

Understanding how people perceive algorithmic decision-making is remains critical, as these systems are increasingly integrated into areas such as education, healthcare, and criminal justice. These perceptions can shape trust in, compliance with, and the perceived legitimacy of automated systems. Focusing on San Francisco's decade-long policy of algorithmic school assignments, we draw on procedural and distributive justice theory to investigate parents' fairness perceptions of San Francisco's school assignment system. We find that key differences in parents' definitions of fairness shape their preferences for what constitutes a fair school assignment system, while also controlling for parents' school assignment outcomes. Moreover, parents' definitions and perceptions of fairness differ across socioeconomic and racial groups. For instance, among white respondents, the most used definition of fairness was "proximity'' to their assigned school, whereas, among Hispanic or Latino parents, the most popular definition of fairness was that the "same rules'' are applied to everyone. It is crucial for computational system designers and policymakers to consider these differences when deciding on the goals and values embedded in decision-making systems and who those goals and values reflect.

Emerging methods for participatory algorithm design have proposed collecting and aggregating individual stakeholders’ preferences to create algorithmic systems that account for those stakeholders’ values. Drawing on two years of research across two public school districts in the United States, we study how families and school districts use students’ preferences for schools to meet their goals in the context of algorithmic student assignment systems. We find that the design of the preference language, i.e. the structure in which participants must express their needs and goals to the decision-maker, shapes the opportunities for meaningful participation. We define three properties of preference languages – expressiveness, cost, and collectivism – and discuss how these factors shape who is able to participate, and the extent to which they are able to effectively communicate their needs to the decision-maker. Reflecting on these findings, we offer implications and paths forward for researchers and practitioners who are considering applying a preference-based model for participation in algorithmic decision making.

A major challenge in the practical use of Machine Translation (MT) is that users lack guidance to make informed decisions about when to rely on outputs.Progress in quality estimation research provides techniques to automatically assess MT quality, but these techniques have primarily been evaluated in vitro by comparison against human judgments outside of a specific context of use.This paper evaluates quality estimation feedback in vivo with a human study simulating decision-making in high-stakes medical settings.Using Emergency Department discharge instructions, we study how interventions based on quality estimation versus backtranslation assist physicians in deciding whether to show MT outputs to a patient.We find that quality estimation improves appropriate reliance on MT, but backtranslation helps physicians detect more clinically harmful errors that QE alone often misses.

Social media platforms are celebrated for their capacity to empower those with marginalized or disenfranchised identities and support them to create counterpublics. We focus on one such group, Muslim Americans, and ask how visible Muslim Americans, such as journalists, activists, and aspiring politicians, use social media to craft counter-narratives, reclaim control of their stories, and mitigate the harm directed at them. Through a series of 19 interviews, we found that visible Muslim Americans' ability to craft and sustain counter narratives is largely hampered by sustained online harm (e.g. harassment). We found that these public figures were harmed repeatedly over long periods of time and through the weaponization of platform affordances such as replying, tagging, and hashtag takeovers, as well as the weaponization of their gender and identity. Our findings shed light on the serious limitations of social media to provide a safe platform for counterpublics to engage externally with wider publics. Finally, we discuss the limitations of content moderation as the dominant framework for addressing harm online and suggest alternative paths forward based on restorative and transformative justice.

Most social media platforms implement content moderation to address interpersonal harms such as harassment. Content moderation relies on offender-centered, punitive approaches, e.g., bans and content removal. We consider an alternative justice framework, restorative justice, which aids victims in healing, supports offenders in repairing the harm, and engages community members in addressing the harm collectively. To assess the utility of restorative justice in addressing online harm, we interviewed 23 users from Overwatch gaming communities, including moderators, victims, and offenders; such communities are particularly susceptible to harm, with nearly three quarters of all online game players suffering from some form of online abuse. We study how the communities currently handle harm cases through the lens of restorative justice and examine their attitudes toward implementing restorative justice processes. Our analysis reveals that cultural, technical, and resource-related obstacles hinder implementation of restorative justice within the existing punitive framework despite online community needs and existing structures to support it. We discuss how current content moderation systems can embed restorative justice goals and practices and overcome these challenges.

Online harm is a prevalent issue in adolescents’ online lives. Restorative justice teaches us to focus on those who have been harmed, ask what their needs are, and engage in the offending party and community members to collectively address the harm. In this research, we conducted interviews and design activities with harmed adolescents to understand their needs to address online harm. They also identified the key stakeholders relevant to their needs, the desired outcomes, and the preferred timing to achieve them. We identified five central needs of harmed adolescents: sensemaking, emotional support and validation, safety, retribution, and transformation. We find that addressing the needs of those who are harmed online usually requires concerted efforts from multiple stakeholders online and offline. We conclude by discussing how platforms can implement design interventions to meet some of these needs.

Targeted advertising can harm vulnerable groups when it targets individuals' personal and psychological vulnerabilities. We focus on how targeted weight-loss advertisements harm people with histories of disordered eating. We identify three features of targeted advertising that cause harm: the persistence of personal data that can expose vulnerabilities, over-simplifying algorithmic relevancy models, and design patterns encouraging engagement that can facilitate unhealthy behavior. Through a series of semi-structured interviews with individuals with histories of unhealthy body stigma, dieting, and disordered eating, we found that targeted weight-loss ads reinforced low self-esteem and deepened pre-existing anxieties around food and exercise. At the same time, we observed that targeted individuals demonstrated agency and resistance against distressing ads. Drawing on scholarship in postcolonial environmental studies, we use the concept of slow violence to articulate how online targeted advertising inflicts harms that may not be immediately identifiable. CAUTION: This paper includes media that could be triggering, particularly to people with an eating disorder. Please use caution when reading, printing, or disseminating this paper.

Critical refusal is an active process; an informed practice of investigating power differences in order to generate more just and equitable alternatives to the status quo. In this paper, we examine what it means to utilize critical refusal as a tool for investigating unequal power dynamics that are produced and reified by data practices. We illustrate the generative capacity of critical refusal by drawing on declarations from The Feminist Data Manifest-No to examine data practices across three real-world cases. By pairing a conceptual exploration of critical refusal with real-world examples, we make a theoretical contribution that is grounded in concrete approaches for generating alternative data practices in ways that account for interlocking struggles across contexts and communities.

Instagram infographics are a digital activism tool that have redefined action frames for technology-facilitated social movements. From the 1960s through the 1980s, United States ethnic movements practiced collective action: ideologically unified, resource-intensive activism. Researchers have argued that modern technologically mediated movements, in contrast, practice connective action: individualized, low-resource online activism. We argue that Instagram infographics are both connective and collective. We conducted a qualitative interview study juxtaposing the insights of past and present U.S. ethnic movement activists and analyzed Black Lives Matter Instagram data over the course of 7 years (2014-2020). We find that Instagram infographic activism bridges connective and collective action in three ways: (1) Scope for Education: Visually enticing and digestible infographics reduce the friction of information dissemination, facilitating collective movement education while preserving customizability. (2) Reconciliation for Credibility: Activists use connective features to combat infographic misinformation and resolve internal differences, creating a trusted collective movement front. (3) High-Resource Efforts for Transformative Change: Instagram infographic activism has been paired with boots on the ground and action-oriented content, curating a connective-to-collective pipeline that expends movement resources. Our work unveils the vitality of evaluating digital activism action frames at the movement integration level, exemplifies the powerful coexistence of connective and collective action, and offers design implications for activists seeking to leverage this novel tool.

How do power dynamics and value conflicts affect our ability to design and maintain socio-technical algorithmic processes? In this paper, we study the SIGCHI student volunteer (SV) selection process that uses a weighted semi-randomized algorithm to recruit a desired pool of volunteers. Our interviews with the community members showed that the process is complex and socio-technical; the algorithm's outputs are interpreted and adjusted by the conference organizers to reflect the community values while ensuring the selection of effective volunteers to help with organizing the conference. This provides a stage in which the power dynamics and value conflicts among the stakeholders play salient roles in determining how the process was perceived and envisioned. For instance, non-organizers of the conference found the algorithm used in the selection process to be a power-balancer that places a check on the organizers who oversee the process. However, even with a participatory process to elicit the algorithm's weights, the power dynamics and value conflicts between the participants made it difficult to reach a consensus on what the SV selection process should consider and prioritize. Our findings highlight the importance of value transparency -- the type of transparency that focuses on explaining why a decision was made rather than how it was made -- as a mechanism for resolving such conflicts. Based on our findings, we lay out design recommendations that can guide communities to better design and maintain algorithmic socio-technical processes over time in the face of power dynamics and value conflicts.

Public school districts across the United States have implemented school choice systems that have the potential to improve underserved students' access to educational opportunities. However, research has shown that learning about and applying for schools can be extremely time-consuming and expensive, making it difficult for these systems to create more equitable access to resources in practice. A common factor surfaced in prior work is unequal access to information about the schools and enrollment process. In response, governments and non-profits have invested in providing more information about schools to parents, for instance, through detailed online dashboards. However, we know little about what information is actually useful for historically marginalized and underserved families. We conducted interviews with 10 low-income families and families of color to learn about the challenges they faced navigating an online school choice and enrollment system. We complement this data with four interviews with people who have supported families through the enrollment process in a wide range of roles, from school principal to non-profit staff ("parent advocates''). Our findings highlight the value of personalized support and trusting relationships to delivering relevant and helpful information. We contrast this against online information resources and dashboards, which tend to be impersonal, target a broad audience, and make strong assumptions about what parents should look for in a school without sensitivity to families' varying circumstances. We advocate for an assets-based design approach to information support in public school enrollment, which would ask how we can support the local, one-on-one support that community members already provide.

In the United States, public defenders (lawyers assigned to people accused of crimes who cannot afford a private attorney) serve as an essential bulwark against wrongful arrest and incarceration for low-income and marginalized people. Public defenders have long been overworked and under-resourced. However, these issues have been compounded by increases in the volume and complexity of data in modern criminal cases. We explore the technology needs of public defenders through a series of semi-structured interviews with public defenders and those who work with them. We find that public defenders' ability to reason about novel surveillance data is woefully inadequate not only due to a lack of resources and knowledge, but also due to the structure of the criminal justice system, which gives prosecutors and police (in partnership with private companies) more control over the type of information used in criminal cases than defense attorneys. We find that public defenders may be able to create fairer situations for their clients with better tools for data interpretation and access. Therefore, we call on technologists to attend to the needs of public defenders and the people they represent when designing systems that collect data about people. Our findings illuminate constraints that technologists and privacy advocates should consider as they pursue solutions. In particular, our work complicates notions of individual privacy as the only value in protecting users' rights, and demonstrates the importance of data interpretation alongside data visibility. As data sources become more complex, control over the data cannot be separated from access to the experts and technology to make sense of that data. The growing surveillance data ecosystem may systematically oppress not only those who are most closely observed, but groups of people whose communities and advocates have been deprived of the storytelling power over their information.

Across the United States, a growing number of school districts are turning to matching algorithms to assign students to public schools. The designers of these algorithms aimed to promote values such as transparency, equity, and community in the process. However, school districts have encountered practical challenges in their deployment. In fact, San Francisco Unified School District voted to stop using and completely redesign their student assignment algorithm because it was frustrating for families and it was not promoting educational equity in practice. We analyze this system using a Value Sensitive Design approach and find that one reason values are not met in practice is that the system relies on modeling assumptions about families’ priorities, constraints, and goals that clash with the real world. These assumptions overlook the complex barriers to ideal participation that many families face, particularly because of socioeconomic inequalities. We argue that direct, ongoing engagement with stakeholders is central to aligning algorithmic values with real world conditions. In doing so we must broaden how we evaluate algorithms while recognizing the limitations of purely algorithmic solutions in addressing complex socio-political problems.

Efforts to make machine learning more widely accessible have led to a rapid increase in Auto-ML tools that aim to automate the process of training and deploying machine learning. To understand how Auto-ML tools are used in practice today, we performed a qualitative study with participants ranging from novice hobbyists to industry researchers who use Auto-ML tools. We present insights into the benefits and deficiencies of existing tools, as well as the respective roles of the human and automation in ML workflows. Finally, we discuss design implications for the future of Auto-ML tool development. We argue that instead of full automation being the ultimate goal of Auto-ML, designers of these tools should focus on supporting a partnership between the user and the Auto-ML tool. This means that a range of Auto-ML tools will need to be developed to support varying user goals such as simplicity, reproducibility, and reliability.

Among many young people, the creation of a finsta-a portmanteau of "fake" and "Instagram" which describes secondary Instagram accounts-provides an outlet to share emotional, low-quality, or indecorous content with their close friends. To study why people create and maintain finstas, we conducted a qualitative study through interviews with finsta users and content analysis of video bloggers exposing their finsta on YouTube. We found that one way that young people deal with mounting social pressures is by reconfiguring online platforms and changing their purposes, norms, expectations, and currencies. Carving out smaller spaces accessible only to close friends allows users the opportunity for a more unguarded, vulnerable, and unserious performance. Drawing on feminist theory, we term this process intimate reconfiguration. Through this reconfiguration finsta users repurpose an existing and widely-used social platform to create opportunities for more meaningful and reciprocal forms of social support.

This joint panel between UIST and CSCW brings together leading researchers at the intersection of the conferences-systems researchers in collaborative and social computing-to engage in a discussion and retrospective. Pairs of panelists will represent each decade since the founding of the conferences, sharing a brief retrospective that surveys the most influential papers of that decade, the zeitgeist of the problems that were popular that decade and why, and what each decade's work has to say to the decades that came before and after. The panel is intended as a space to celebrate advances in the field, and reflect on the burdens and opportunities that it faces ahead.

Online content creators have to manage their relations with opaque, proprietary algorithms that platforms employ to rank, filter, and recommend content. How do content creators make sense of these algorithms and what does that teach us about the roles that algorithms play in the social world? We take the case of YouTube because of its widespread use and the spaces for collective sense-making and mutual aid that content creators (YouTubers) have built within the last decade. We engaged with YouTubers in one-on-one interviews, performed content analysis on YouTube videos that discuss the algorithm, and conducted a wiki survey on YouTuber online groups. This triangulation of methodologies afforded us a rich understanding of content creators' understandings, priorities, and wishes as they relate to the algorithm. We found that YouTubers assign human characteristics to the algorithm to explain its behavior; what we have termed algorithmic personas. We identify three main algorithmic personas on YouTube: Agent, Gatekeeper, and Drug Dealer. We propose algorithmic personas as a conceptual framework that describes the new roles that algorithmic systems take on in the social world. As we face new challenges around the ethics and politics of algorithmic platforms such as YouTube, algorithmic personas describe roles that are familiar and can help develop our understanding of algorithmic power relations and accountability mechanisms.

Collectives gather online around challenges they face, but frequently fail to envision shared outcomes to act on together. Prior work has developed systems for improving collective ideation and design by exposing people to each others' ideas and encouraging them to intermix those ideas. However, organizational behavior research has demonstrated that intermixing ideas does not result in meaningful engagement with those ideas. In this paper, we introduce a new class of collective design system that intermixes people instead of ideas: instead of receiving mere exposure to others' ideas, participants engage deeply with other members of the collective who represent those ideas, increasing engagement and influence. We thus present Hive: a system that organizes a collective into small teams, then intermixes people by rotating team membership over time. At a technical level, Hive must balance two competing forces: (1) networks are better at connecting diverse perspectives when network efficiency is high, but (2) moving people diminishes tie strength within teams. Hive balances these two needs through network rotation: an optimization algorithm that computes who should move where, and when. A controlled study compared network rotation to alternative rotation systems which maximize only tie strength or network efficiency, finding that network rotation produced higher-rated proposals. Hive has been deployed by Mozilla for a real-world open design drive to improve Firefox accessibility.

The web affords connections by which end-users can receive paid, expert help—such as programming, design, and writing—to reach their goals. While a number of online marketplaces have emerged to facilitate such connections, most end-users do not approach a market to hire an expert when faced with a challenge. To reduce friction in hiring from peer-to-peer expert crowd work markets, we propose Ink, a system that crowd workers can use to showcase their services by embedding tasks inside web tutorials—a common destination for users with information needs. Workers have agency to define and manage tasks, through which users can request their help to review or execute each step of the tutorial, for example, to give feedback on a paper outline, perform a statistical analysis, or host a practice programming interview. In a public deployment, over 25,000 pageviews led 168 tutorial readers to pay crowd workers for their services, most of whom had not previously hired from crowdsourcing marketplaces. A field experiment showed that users were more likely to hire crowd experts when the task was embedded inside the tutorial rather than when they were redirected to the same worker’s Upwork profile to hire them. Qualitative analysis of interviews showed that Ink framed hiring expert crowd workers within users’ well-established information seeking habits and gave workers more control over their work.

Distributed, parallel crowd workers can accomplish simple tasks through workflows, but teams of collaborating crowd workers are necessary for complex goals. Unfortunately, a fundamental condition for effective teams -- familiarity with other members -- stands in contrast to crowd work's flexible, on-demand nature. We enable effective crowd teams with Huddler, a system for workers to assemble familiar teams even under unpredictable availability and strict time constraints. Huddler utilizes a dynamic programming algorithm to optimize for highly familiar teammates when individual availability is unknown. We first present a field experiment that demonstrates the value of familiarity for crowd teams: familiar crowd teams doubled the performance of ad-hoc (unfamiliar) teams on a collaborative task. We then report a two-week field deployment wherein Huddler enabled crowd workers to convene highly familiar teams in 18 minutes on average. This research advances the goal of supporting long-term, team-based collaborations without sacrificing the flexibility of crowd work.

Crowd work is typically limited to simple, context-free tasks because they are easy to describe and understand. In contrast, complex tasks require communication between the requester and workers to achieve mutual understanding, which can be more work than it is worth. This paper explores the notion of structured communication: using structured microtasks to support communication in the domain of complex writing. Our studies compare a variety of communication mechanisms with respect to the costs to the requester in providing information and the value of that information to workers while performing the task. We find that different mechanisms are effective at different stages of writing. For early drafts, asking the requester to state the biggest problem in the current write-up is valuable and low cost, while later it is more useful for the worker if the requester highlights the text that needs to be improved. These findings can be used to enable richer, more interactive crowd work than what currently seems possible. We incorporate the findings in a workflow for crowdsourcing written content using appropriately timed mechanisms for communicating with the crowd.

Expert crowdsourcing marketplaces have untapped potential to empower workers' career and skill development. Currently, many workers cannot afford to invest the time and sacrifice the earnings required to learn a new skill, and a lack of experience makes it difficult to get job offers even if they do. In this paper, we seek to lower the threshold to skill development by repurposing existing tasks on the marketplace as mentored, paid, real-world work experiences, which we refer to as micro-internships. We instantiate this idea in Atelier, a micro-internship platform that connects crowd interns with crowd mentors. Atelier guides mentor-intern pairs to break down expert crowdsourcing tasks into milestones, review intermediate output, and problem-solve together. We conducted a field experiment comparing Atelier's mentorship model to a non-mentored alternative on a real-world programming crowdsourcing task, finding that Atelier helped interns maintain forward progress and absorb best practices.

By lowering the costs of communication, the web promises to enable distributed collectives to act around shared issues. However, many collective action efforts never succeed: while the web's affordances make it easy to gather, these same decentralizing characteristics impede any focus towards action. In this paper, we study challenges to collective action efforts through the lens of online labor by engaging with Amazon Mechanical Turk workers. Through a year of ethnographic fieldwork, we sought to understand online workers' unique barriers to collective action. We then created Dynamo, a platform to support the Mechanical Turk community in forming publics around issues and then mobilizing. We found that collective action publics tread a precariously narrow path between the twin perils of stalling and friction, balancing with each step between losing momentum and flaring into acrimony. However, specially structured labor to maintain efforts' forward motion can help such publics take action.

The growing use of social media means that an increasing amount of people's lives are visible online. We draw from Goffman's theatrical metaphor and Hogan's exhibition approach to explore how people manage their personal collection of social media data over time. We conducted a qualitative study of 13 participants to reveal their day-to-day decision-making about producing and curating digital traces on Facebook. Their goals and strategies showed that people experience the Facebook platform as consisting of three different functional regions: a performance region for managing recent data and impression management, an exhibition region for longer term presentation of self-image, and a personal region for archiving meaningful facets of life. Further, users' need for presenting and archiving data in these three regions is mediated by temporality. These findings trigger a discussion of how to design social media that support these dynamic and sometimes conflicting needs.