As misinformation proliferates online, large language models (LLMs) have been proposed as a promising tool to accelerate fact-checking workflows. While LLMs demonstrate strong performance in tasks such as text annotation, their capabilities in generating fact-checking reports remain uncertain. To investigate how media experts evaluate LLM-generated fact-checking reports, we conducted a 2 (Source: human vs. LLM) X 2 (Disclosure of Source: yes or no) between-subjects online experiment with media professionals (N=274). Our analyses reveal that experts perceive LLM-generated reports as significantly less useful than human-written reports; and such differences become larger when participants are not aware of the source. However, LLM-generated fact-checking reports were rated as accurate and logical as human-authored ones. Party affiliation plays a role in predicting perceived logicalness. Our findings advance the understanding of experts’ evaluation of LLM-generated content within the context of misinformation, which provides important theoretical contributions to HCI and communication theories as well as practical implications for the field.

Despite the evidence of harm that technology can inflict, commensurate policymaking to hold tech platforms accountable still lags. This is pertinent to app-based gig workers, where unregulated algorithms continue to dictate their work, often with little human recourse. While past HCI literature has investigated workers’ experiences under algorithmic management and how to design interventions, rarely are the perspectives of stakeholders who inform or craft policy sought. To bridge this, we propose using data probes—interactive visualizations of workers’ data that show the impact of technology practices on people—exploring them in 12 semi-structured interviews with policy informers, (driver-)organizers, litigators, and a lawmaker in the rideshare space. We show how data probes act as boundary objects to assist stakeholder interactions, demystify technology for policymakers, and support worker collective action. We discuss the potential for data probes as training tools for policymakers, and considerations around data access and worker risks when using data probes.

While many Natural Language Processing (NLP) techniques have been proposed for fact-checking, both academic research and fact-checking organizations report limited adoption of such NLP work due to poor alignment with fact-checker practices, values, and needs. To address this, we investigate a co-design method, Matchmaking for AI, to enable fact-checkers, designers, and NLP researchers to collaboratively identify what fact-checker needs should be addressed by technology, and to brainstorm ideas for potential solutions. Co-design sessions we conducted with 22 professional fact-checkers yielded a set of 11 design ideas that offer a "north star'', integrating fact-checker criteria into novel NLP design concepts. These concepts range from pre-bunking misinformation, efficient and personalized monitoring misinformation, proactively reducing fact-checker potential biases, and collaborative writing fact-check reports. Our work provides new insights into both human-centered fact-checking research and practice and AI co-design research.

AI technologies continue to advance from digital assistants to assisted decision-making. However, designing AI remains a challenge given its unknown outcomes and uses. One way to expand AI design is by centering stakeholders in the design process. We conduct co-design sessions with gig workers to explore the design of gig worker-centered tools as informed by their driving patterns, decisions, and personal contexts. Using workers’ own data as well as city-level data, we create probes—interactive data visuals—that participants explore to surface the well-being and positionalities that shape their work strategies. We describe participant insights and corresponding AI design considerations surfaced from data probes about: 1) workers’ well-being trade-offs and positionality constraints, 2) factors that impact well-being beyond those in the data probes, and 3) instances of unfair algorithmic management. We discuss the implications for designing data probes and using them to elevate worker-centered AI design as well as for worker advocacy.

Prior research has studied the detrimental impact of algorithmic management on gig workers and strategies that workers devise in response. However, little work has investigated alternative platform designs to promote worker well-being, particularly from workers’ own perspectives. We use a participatory design approach wherein workers explore their algorithmic imaginaries to co-design interventions that center their lived experiences, preferences, and well-being in algorithmic management. Our interview and participatory design sessions highlight how various design dimensions of algorithmic management, including information asymmetries and unfair, manipulative incentives, hurt worker well-being. Workers generate designs to address these issues while considering competing interests of the platforms, customers, and themselves, such as information translucency, incentives co-configured by workers and platforms, worker-centered data-driven insights for well-being, and collective driver data sharing. Our work offers a case study that responds to a call for designing worker-centered digital work and contributes to emerging literature on algorithmic work.

Hyper-partisan misinformation has become a major public concern. In order to examine what type of misinformation label can mitigate hyper-partisan misinformation sharing on social media, we conducted a 4 (label type: algorithm, community, third-party fact-checker, and no label) X 2 (post ideology: liberal vs. conservative) between-subjects online experiment (N = 1,677) in the context of COVID-19 health information. The results suggest that for liberal users, all labels reduced the perceived accuracy and believability of fake posts regardless of the posts' ideology. In contrast, for conservative users, the efficacy of the labels depended on whether the posts were ideologically consistent: algorithmic labels were more effective in reducing the perceived accuracy and believability of fake conservative posts compared to community labels, whereas all labels were effective in reducing their belief in liberal posts. Our results shed light on the differing effects of various misinformation labels dependent on people's political ideology.