In this work, we empirically evaluated expert-led debunking of health misinformation on TikTok with n=420 survey and n=20 interview participants. Unlike fact-checkers, health professionals debunk misinformation non-anonymously through video-against-video formats (stitching/”duetting”), rather than using labels. We analyzed 5,161 such posts to select six misinformation and six debunking videos across three common topics – two general health, two mental health, and two nutrition – for statistical comparison. Participants exposed to debunking videos believed misinformation claims significantly less than those exposed to misinformation videos in all six conditions. Experts were seen as more credible than misinformation creators, except in one instance involving mental health. Thematic analysis showed that expert-led debunking succeeded because experts’ videos aligned with the Debunking Handbook method for effective refutation. Experts’ credibility is derived mainly from being perceived as non-typical influencers who maintain reputable TikTok personas by providing qualified medical evidence and advice.

Audio deepfakes are a form of deception where convincing speech sentences are synthesized through machine learning means to give an impression of a human speaker. Audio deepfakes emerge as an attractive vector for targeting users that rely on audio accessibility, such as individuals who are blind or low vision. The critical reliance on speech both as a medium and an affordance puts this population at an undue risk of being deceived as they rely solely on themselves to detect whether a piece of audio is a deepfake or not. To better understand the nature of this risk considering the nuanced reliance on assistive technologies such as screen readers, we conducted a user study with n=16 blind and low vision individuals from the US. Our participants achieved an overall discernment accuracy of 59%, and clips identified as deep fakes were only actually deepfakes in 50.8% of the cases (precision). The participants that self-identified as "low vision" performed slightly better (accuracy of 61%, precision of 64%) compared to the ones that self-identified as "blind" (accuracy of 55%, precision of 56%). Our qualitative results show that the participants in the "blind" group mostly considered a combination of infliction, imperfections in the voice, and the intensity in the speech delivery as discernment factors. The participants in the "low vision" group mostly used the speaker's pitch, enunciation, emotion, and the fluency and articulation of the speaker as discernment cues. Overall, participants felt that audio deepfakes have the potential to deceive visually impaired individuals with political disinformation, impersonate their voice in authentication and smart homes, and specifically target them with voice phishing and enhanced scams.

Abstract This paper reports the findings from an empirical study investigating the effectiveness of using intelligent voice assistants, Amazon Alexa in our case, to deliver a phishing training to users. Because intelligent voice assistants can hardly utilize visual cues but provide for convenient interaction with users, we developed an interaction-based phishing training focused on the principles of persuasion with examples on how to look for them in phishing emails. To test the effectiveness of this training, we conducted a between-subject study where 120 participants were randomly assigned in three groups: no training, interaction-based training with Alexa, and a facts-and-advice training and assessed a vignette of 28 emails. The results show that the participants in the interaction-based group statistically outperformed the others when detecting phishing emails that employed the following persuasion principles (and/or combinations of): authority, authority/scarcity, commitment, commitment/liking, and scarcity/liking. The paper discusses the implication of this result for future phishing training and anti-phishing efforts.

This paper reports the findings of a study testing a novel social engineering attack called Malware-induced Misperception (MIM). The goal of the MIM attack is to induce misperception by using a man-in-the-middle malware that covertly rearranges the linguistic content of an authentic social media post, web page, or an e-mail. The MIM attack was tested in controlled settings (N=311) where the malware covertly manipulated the linguistic content of a Facebook discourse to induce misperception about the climate of opinion on a polarizing issue (freedom of speech on college campuses) by making conservative-leaning comments appear as liberal-leaning. The induced misperception was assessed in the context of the spiral-of-silence theory. The theory predicts that polarizing issues discourage commenting on social media if an individual’s opinion diverges from the perceived climate of opinion on that issue. Consistent with the theory, the results suggest that the MIM attack can socially engineer the spiral-of-silence effect by manipulating the comments in a Facebook discourse to appeal to the individual’s political ideology and gender identity.Additional Key Words and Phrases: Malware-Induced Misperception (MIM); spiral-of-silence; social engineering; Facebook, web security

VoxPop, shortened for Vox Populi, is an experimental social media platform that neither has an absolute “truth-keeping” mission nor an uncontrolled “free-speaking” vision. Instead, it allows discourses that naturally include (mis)information to contextualize among users with the aid of UX design and data science affordances and frictions. VoxPop introduces calibration metrics, namely a Faithfulness-To-Known-Facts (FTKF) score associated with each post and a Cumulative FTKF (C-FTKF) score associated with each user, appealing to the self-regulated participation using sociocognitive signals. The goal of VoxPop is not to become an ideal platform—that is impossible; rather, to bring to attention an adaptive approach in dealing with (mis)information rooted in social calibration instead of imposing or avoiding altogether punitive moderation.

Trolling and social bots have been proven as powerful tactics for manipulating the public opinion and sowing discord among Twitter users. This effort requires substantial content fabrication and account coordination to evade Twitter's detection of nefarious platform use. In this paper we explore an alternative tactic for covert social media interference by inducing misperceptions about genuine, non-trolling content from verified users. This tactic uses a malware that covertly manipulates targeted words, hashtags, and Twitter metrics before the genuine content is presented to a targeted user in a covert man-in-the-middle fashion. Early tests of the malware found that it is capable of achieving a similar goal as trolls and social bots, that is, silencing or provoking social media users to express their opinion in polarized debates on social media. Following this, we conducted experimental tests in controlled settings (N = 315) where the malware covertly manipulated the perception in a Twitter debate on the risk of vaccines causing autism. The empirical results demonstrate that inducing misperception is an effective tactic to silence users on Twitter when debating polarizing issues like vaccines. We used the findings to propose a solution for countering the effect of the malware-induced misperception that could also be used against trolls and social bots on Twitter.

This paper presents TrollHunter, an automated reasoning mechanism we used to hunt for trolls on Twitter during the COVID-19 pandemic in 2020. Trolls quickly seized the opportunity to create a COVID-19 infodemic by promulgating dubious content on Twitter. To counter the COVID-19 infodemic, the TrollHunter leverages a unique linguistic analysis of a multi-dimensional set of Twitter content features to detect whether or not a tweet was meant to troll. TrollHunter achieved 98.5% accuracy, 75.4% precision and 69.8% recall over a dataset of 1.3 million tweets. Without a final resolution of the pandemic in sight, it is unlikely that the trolls will go away, although they might be forced to evade automated hunting. To explore the plausibility of this strategy, we developed and tested an adversarial machine learning mechanism called TrollHunter-Evader. TrollHunter-Evader employs a Test Time Evasion (TTE) approach in a combination with a Markov chain-based mechanism to recycle originally trolling tweets. The recycled tweets were able to achieve a remarkable 40% decrease in the TrollHunter’s ability to correctly identify trolling tweets. Because the COVID-19 infodemic could have a harmful impact on the COVID-19 pandemic, we provide an elaborate discussion about the implications of employing adversarial machine learning to evade Twitter troll hunts.