The growing presence of AI-manipulated videos presents a significant challenge to the integrity of online information. This paper presents findings from an empirical study with 490 participants in the United States to provide a holistic view of public engagement with this threat. We structure our analysis around three key areas: (1) how demographics and media habits influence general perceptions of prevalence; (2) the factors shaping detection accuracy, the calibration of confidence level, and the perceptual cues people rely on when viewing in-the-wild videos; and (3) the verification actions people take following suspicion. We find that while the public views AI-manipulated media as prevalent, participants struggled to distinguish authentic and AI-manipulated videos, often exhibiting poorly calibrated confidence. Furthermore, users rarely utilize available detection tools. These patterns highlight the insufficiency of human detection ability and the need of new approaches to enable improved user awareness, successful interventions, and effective mitigation.

organize analytics Figure 1: Study flow showing the two phases of user studies: the Requirements Study, followed by the Ontology Study and their sub-steps.

The evolving landscape of manipulated media, including the threat of deepfakes, has made information verification a daunting challenge for journalists. Technologists have developed tools to detect deepfakes, but these tools can sometimes yield inaccurate results, raising concerns about inadvertently disseminating manipulated content as authentic news. This study examines the impact of unreliable deepfake detection tools on information verification. We conducted role-playing exercises with 24 US journalists, immersing them in complex breaking-news scenarios where determining authenticity was challenging. Through these exercises, we explored questions regarding journalists’ investigative processes, use of a deepfake detection tool, and decisions on when and what to publish. Our findings reveal that journalists are diligent in verifying information, but sometimes rely too heavily on results from deepfake detection tools. We argue for more cautious release of such tools, accompanied by proper training for users to mitigate the risk of unintentionally propagating manipulated content as real news.