As misinformation proliferates online, large language models (LLMs) have been proposed as a promising tool to accelerate fact-checking workflows. While LLMs demonstrate strong performance in tasks such as text annotation, their capabilities in generating fact-checking reports remain uncertain. To investigate how media experts evaluate LLM-generated fact-checking reports, we conducted a 2 (Source: human vs. LLM) X 2 (Disclosure of Source: yes or no) between-subjects online experiment with media professionals (N=274). Our analyses reveal that experts perceive LLM-generated reports as significantly less useful than human-written reports; and such differences become larger when participants are not aware of the source. However, LLM-generated fact-checking reports were rated as accurate and logical as human-authored ones. Party affiliation plays a role in predicting perceived logicalness. Our findings advance the understanding of experts’ evaluation of LLM-generated content within the context of misinformation, which provides important theoretical contributions to HCI and communication theories as well as practical implications for the field.

The wide adoption of platformized work has generated remarkable advancements in the labor patterns and mobility of modern society. Underpinning such progress, gig workers are exposed to unprecedented challenges and accountabilities: lack of data transparency, social and physical isolation, as well as insufficient infrastructural safeguards. Gig2Gether presents a space designed for workers to engage in an initial experience of voluntarily contributing anecdotal and statistical data to affect policy and build solidarity across platforms by exchanging unifying and diverse experiences. Our 7-day field study with 16 active workers from three distinct platforms and work domains showed existing affordances of data-sharing: facilitating mutual support across platforms, as well as enabling financial reflection and planning. Additionally, workers envisioned future use cases of data-sharing for collectivism (e.g., collaborative examinations of algorithmic speculations) and informing policy (e.g., around safety and pay), which motivated (latent) worker desiderata of additional capabilities and data metrics. Based on these findings, we discuss remaining challenges to address and how data-sharing tools can complement existing structures to maximize worker empowerment and policy impact.

City governments in the United States are increasingly pressured to adopt emerging technologies. Yet, these systems often risk biased and disparate outcomes. Scholars studying public sector technology design have converged on the need to ground these systems in the goals and organizational contexts of employees using them. We expand our understanding of employees' contexts by focusing on the equity practices of city government employees to surface important equity considerations around public sector data and technology use. Through semi-structured interviews with thirty-six employees from ten departments of a U.S. city government, our findings reveal challenges employees face when operationalizing equity, perspectives on data needs for advancing equity goals, and the design space for acceptable government technology. We discuss what it looks like to foreground equity in data use and technology design, and considerations for how to support city government employees in operationalizing equity with and without official equity offices.

Despite the evidence of harm that technology can inflict, commensurate policymaking to hold tech platforms accountable still lags. This is pertinent to app-based gig workers, where unregulated algorithms continue to dictate their work, often with little human recourse. While past HCI literature has investigated workers’ experiences under algorithmic management and how to design interventions, rarely are the perspectives of stakeholders who inform or craft policy sought. To bridge this, we propose using data probes—interactive visualizations of workers’ data that show the impact of technology practices on people—exploring them in 12 semi-structured interviews with policy informers, (driver-)organizers, litigators, and a lawmaker in the rideshare space. We show how data probes act as boundary objects to assist stakeholder interactions, demystify technology for policymakers, and support worker collective action. We discuss the potential for data probes as training tools for policymakers, and considerations around data access and worker risks when using data probes.

AI technologies continue to advance from digital assistants to assisted decision-making. However, designing AI remains a challenge given its unknown outcomes and uses. One way to expand AI design is by centering stakeholders in the design process. We conduct co-design sessions with gig workers to explore the design of gig worker-centered tools as informed by their driving patterns, decisions, and personal contexts. Using workers’ own data as well as city-level data, we create probes—interactive data visuals—that participants explore to surface the well-being and positionalities that shape their work strategies. We describe participant insights and corresponding AI design considerations surfaced from data probes about: 1) workers’ well-being trade-offs and positionality constraints, 2) factors that impact well-being beyond those in the data probes, and 3) instances of unfair algorithmic management. We discuss the implications for designing data probes and using them to elevate worker-centered AI design as well as for worker advocacy.

Research exploring how to support decision-making has often used machine learning to automate or assist human decisions. We take an alternative approach for improving decision-making, using machine learning to help stakeholders surface ways to improve and make fairer decision-making processes. We created "Deliberating with AI", a web tool that enables people to create and evaluate ML models in order to examine strengths and shortcomings of past decision-making and deliberate on how to improve future decisions. We apply this tool to a context of people selection, having stakeholders---decision makers (faculty) and decision subjects (students)---use the tool to improve graduate school admission decisions. Through our case study, we demonstrate how the stakeholders used the web tool to create ML models that they used as boundary objects to deliberate over organization decision-making practices. We share insights from our study to inform future research on stakeholder-centered participatory AI design and technology for organizational decision-making.

Prior research has studied the detrimental impact of algorithmic management on gig workers and strategies that workers devise in response. However, little work has investigated alternative platform designs to promote worker well-being, particularly from workers’ own perspectives. We use a participatory design approach wherein workers explore their algorithmic imaginaries to co-design interventions that center their lived experiences, preferences, and well-being in algorithmic management. Our interview and participatory design sessions highlight how various design dimensions of algorithmic management, including information asymmetries and unfair, manipulative incentives, hurt worker well-being. Workers generate designs to address these issues while considering competing interests of the platforms, customers, and themselves, such as information translucency, incentives co-configured by workers and platforms, worker-centered data-driven insights for well-being, and collective driver data sharing. Our work offers a case study that responds to a call for designing worker-centered digital work and contributes to emerging literature on algorithmic work.

Hyper-partisan misinformation has become a major public concern. In order to examine what type of misinformation label can mitigate hyper-partisan misinformation sharing on social media, we conducted a 4 (label type: algorithm, community, third-party fact-checker, and no label) X 2 (post ideology: liberal vs. conservative) between-subjects online experiment (N = 1,677) in the context of COVID-19 health information. The results suggest that for liberal users, all labels reduced the perceived accuracy and believability of fake posts regardless of the posts' ideology. In contrast, for conservative users, the efficacy of the labels depended on whether the posts were ideologically consistent: algorithmic labels were more effective in reducing the perceived accuracy and believability of fake conservative posts compared to community labels, whereas all labels were effective in reducing their belief in liberal posts. Our results shed light on the differing effects of various misinformation labels dependent on people's political ideology.

Artificial intelligence is increasingly being used to manage the workforce. Algorithmic management promises organizational efficiency, but often undermines worker well-being. How can we computationally model worker well-being so that algorithmic management can be optimized for and assessed in terms of worker well-being? Toward this goal, we propose a participatory approach for worker well-being models. We first define worker well-being models: Work preference models---preferences about work and working conditions, and managerial fairness models---beliefs about fair resource allocation among multiple workers. We then propose elicitation methods to enable workers to build their own well-being models leveraging pairwise comparisons and ranking. As a case study, we evaluate our methods in the context of algorithmic work scheduling with 25 shift workers and 3 managers. The findings show that workers expressed idiosyncratic work preference models and more uniform managerial fairness models, and the elicitation methods helped workers discover their preferences and gave them a sense of empowerment. Our work provides a method and initial evidence for enabling participatory algorithmic management for worker well-being.