The increasing frequency of mass shootings in the United States has become alarmingly common, prompting discussions about gun control. While gun control in the US involves complex legal issues, cultural factors---particularly ``gun culture''---play a significant but often overlooked role. Although the role of social media in shaping culture is well-documented, the intersection of gun culture and fringe online communities, like 4chan, remains unclear. This gap is particularly concerning given the rise in mass shootings and the online radicalization of some shooters. To address this gap, we explore gun culture on /k/, 4chan's weapons board. More specifically, we employ various NLP techniques to analyze over 4M posts on /k/ and contextualize the discussion within the broader body of theoretical framework of gun culture. Our findings reveal that discussions on /k/ cover a wide array of topics, with a significant focus on law-related discussions---over 17% of gun-related conversations on /k/ revolve around legal matters. Additionally, our analysis uncovers the presence of extreme viewpoints surrounding firearms, often manifesting as gun fetishism. These insights can be valuable for a range of stakeholders including social media platform, in efforts to address content moderation and de-radicalization

Online web communities often face bans for violating platform policies, encouraging their migration to alternative platforms. This migration, however, can result in increased toxicity and unforeseen consequences on the new platform. In recent years, researchers have collected data from many alternative platforms, indicating coordinated efforts leading to offline events, conspiracy movements, hate speech propagation, and harassment. Thus, it becomes crucial to characterize and understand these alternative platforms. To advance research in this direction, we collect and release a large-scale dataset from Scored -- an alternative Reddit platform that sheltered banned fringe communities, for example, c/TheDonald (a prominent right-wing community) and c/GreatAwakening (a conspiratorial community). Over four years, we collected approximately 57M posts from Scored, with at least 58 communities identified as migrating from Reddit and over 950 communities created since the platform's inception. Furthermore, we provide sentence embeddings of all posts in our dataset, generated through a state-of-the-art model, to further advance the field in characterizing the discussions within these communities. We aim to provide these resources to facilitate their investigations without the need for extensive data collection and processing efforts.

Alas, coordinated hate attacks, or raids, are becoming increasingly common online. In a nutshell, these are perpetrated by a group of aggressors who organize and coordinate operations on a platform (e.g., 4chan) to target victims on another community (e.g., YouTube). In this paper, we focus on attributing raids to their source community, paving the way for moderation approaches that take the context (and potentially the motivation) of an attack into consideration. We present TUBERAIDER, an attribution system achieving over 75% accuracy in detecting and attributing coordinated hate attacks on YouTube videos. We instantiate it using links to YouTube videos shared on 4chan's /pol/ board, r/The_Donald, and 16 Incels-related subreddits. We use a peak detector to identify a rise in the comment activity of a YouTube video, which signals that an attack may be occurring. We then train a machine learning classifier based on the community language (i.e., TF-IDF scores of relevant keywords) to perform the attribution. We test TUBERAIDER in the wild and present a few case studies of actual aggression attacks identified by it to showcase its effectiveness.

This paper emphasizes the necessity of a precise definition of Islamophobia within the realm of social media platforms. The current broad understanding often leads to misclassification and poses challenges to the principles of freedom of speech. Differentiating between Islamophobia and legitimate criticism presents a complex task for automated hate speech detection models, particularly in the presence of offensive language and emotionally charged tones. Furthermore, the paper highlights the inadvertent discriminatory consequences that can arise from misusing Islamophobia detection models against atheists, feminists, ex-Muslims, and others, underscoring the importance of safeguarding their rights. Our study introduces a refined definition and employs advanced deep learning models. It demonstrates a reduction in the number of Islamophobic comments in the dataset while maintaining the accurate identification of genuine instances of Islamophobia. This distinction is made without compromising discussions related to religion and criticism. The results show promise in improving the precision of Islamophobia identification, all while upholding principles of free expression and open dialogue.

Previous research has documented the existence of both online echo chambers and hostile intergroup interactions. In this paper, we explore the relationship between these two phenomena by studying the activity of 5.97M Reddit users and 421M comments posted over 13 years. We examine whether users who are more engaged in echo chambers are more hostile when they comment on other communities. We then create a typology of relationships between political communities based on whether their users are toxic to each other, whether echo chamber-like engagement with these communities has a polarizing effect, and on the communities' political leanings. We observe both the echo chamber and hostile intergroup interaction phenomena, but neither holds universally across communities. Contrary to popular belief, we find that polarizing and toxic speech is more dominant between communities on the same, rather than opposing, sides of the political spectrum, especially on the left; however, this mostly points to the collective targeting of political outgroups.

The news ecosystem encompasses a wide range of sources with varying levels of trustworthiness, and with public commentary giving different spins to the same stories. In this paper, we present a measurement pipeline able to identify news articles that discuss the same story and trace how they are shared on multiple online communities. We compile a list of 1,073 news websites and extract posts from four Web communities (Twitter, Reddit, 4chan, and Gab) that contain URLs from these sources. This yields a dataset of 38M posts containing 15.6M unique news URLs, spanning almost three years. We study the data along several axes, assessing the trustworthiness of shared news stories, analyzing how they are discussed, and measuring the influence various Web communities have in that. Our analysis shows that different communities discuss different types of news, with polarized communities like Gab and /r/The_Donald subreddit disproportionately referencing untrustworthy sources. We also find t hat f ringe c ommunities o ften h ave a disproportionate influence o n o ther p latforms w .r.t. p ushing n arratives around certain news, for example, about political elections, immigration, or foreign policy. In fact, fringe communities are seemingly successful in influencing the discussion on false narratives about news events on mainstream social networks.

Online fringe communities offer fertile grounds to users seeking and sharing ideas fueling suspicion of mainstream news and conspiracy theories. Among these, the QAnon conspiracy theory emerged in 2017 on 4chan, broadly supporting the idea that powerful politicians, aristocrats, and celebrities are closely engaged in a global pedophile ring. Simultaneously, governments are thought to be controlled by “puppet masters,” as democratically elected officials serve as a fake showroom of democracy.

The outbreak of the COVID-19 pandemic has changed our lives in unprecedented ways. In the face of the projected catastrophic consequences, most countries have enacted social distancing measures in an attempt to limit the spread of the virus. Under these conditions, the Web has become an indispensable medium for information acquisition, communication, and entertainment. At the same time, unfortunately, the Web is being exploited for the dissemination of potentially harmful and disturbing content, such as the spread of conspiracy theories and hateful speech towards specific ethnic groups, in particular towards Chinese people and people of Asian descent since COVID-19 is believed to have originated from China.

Recent progress in genomics has enabled the emergence of a flourishing market for direct-to-consumer (DTC) genetic testing. Companies like 23andMe and AncestryDNA provide affordable health, genealogy, and ancestry reports, and have already tested tens of millions of customers. Consequently, news, experiences, and views on genetic testing are increasingly shared and discussed on social media. At the same time, far-right groups have also taken an interest in genetic testing, using them to attack minorities and prove their genetic “purity.” In this article, we set to study the genetic testing discourse on a number of mainstream and fringe Web communities. We do so in two steps. First, we conduct an exploratory, large-scale analysis of the genetic testing discourse on a mainstream social network such as Twitter. We find that the genetic testing discourse is fueled by accounts that appear to be interested in digital health and technology. However, we also identify tweets with highly racist connotations. This motivates us to explore the connection between genetic testing and racism on platforms with a reputation for toxicity, namely, Reddit and 4chan, where we find that discussions around genetic testing often include highly toxic language expressed through hateful and racist comments. In particular, on 4chan’s politically incorrect board (/pol/), content from genetic testing conversations involves several alt-right personalities and openly anti-semitic rhetoric, often conveyed through memes.

Cyberbullying and cyberaggression are increasingly worrisome phenomena affecting people across all demographics. More than half of young social media users worldwide have been exposed to such prolonged and/or coordinated digital harassment. Victims can experience a wide range of emotions, with negative consequences such as embarrassment, depression, isolation from other community members, which embed the risk to lead to even more critical consequences, such as suicide attempts. In this work, we take the first concrete steps to understand the characteristics of abusive behavior in Twitter, one of today’s largest social media platforms. We analyze 1.2 million users and 2.1 million tweets, comparing users participating in discussions around seemingly normal topics like the NBA, to those more likely to be hate-related, such as the Gamergate controversy, or the gender pay inequality at the BBC station. We also explore specific manifestations of abusive behavior, i.e., cyberbullying and cyberaggression, in one of the hate-related communities (Gamergate). We present a robust methodology to distinguish bullies and aggressors from normal Twitter users by considering text, user, and network-based attributes. Using various state-of-the-art machine-learning algorithms, we classify these accounts with over 90% accuracy and AUC. Finally, we discuss the current status of Twitter user accounts marked as abusive by our methodology and study the performance of potential mechanisms that can be used by Twitter to suspend users in the future.

While developing mobile apps is becoming easier, testing and characterizing their behavior is still hard. On the one hand, the de facto testing tool, called "Monkey," scales well due to being based on random inputs, but fails to gather inputs useful in understanding things like user engagement and attention. On the other hand, gathering inputs and data from real users requires distributing instrumented apps, or even phones with pre-installed apps, an expensive and inherently unscaleable task. To address these limitations we present CHIMP, a system that integrates automated tools and large-scale crowdsourced inputs. CHIMP is different from previous approaches in that it runs apps in a virtualized mobile environment that thousands of users all over the world can access via a standard Web browser. CHIMP is thus able to gather the full range of real-user inputs, detailed run-time traces of apps, and network traffic. We thus describe CHIMP»s design and demonstrate the efficiency of our approach by testing thousands of apps via thousands of crowdsourced users. We calibrate CHIMP with a large-scale campaign to understand how users approach app testing tasks. Finally, we show how CHIMP can be used to improve both traditional app testing tasks, as well as more novel tasks such as building a traffic classifier on encrypted network flows.

Online games provide a rich recording of interactions that can contribute to understanding human behavior. One potential contribution is understanding what motivates people to choose their teammates. We examine several hypotheses about team formation using a large, longitudinal dataset from a team-based online gaming environment. Specifically, we test how positive familiarity, homophily, and competence determine team formation in Battlefield 4, a popular team-based game in which players choose one of two competing teams to play on. Our dataset covers over two months of in-game interactions between over 380,000 players. We show that familiarity is an important factor in team formation, while homophily is not. Competence affects team formation in more nuanced ways: players with similarly high competence team up repeatedly, but large variation in competence discourages repeated interactions.

The theme of cybersafety is an important emerging research topic on the Internet that manifests itself daily as users navigate the Web and networked applications. Examples of cybersafety issues include cyberbullying, cyberthreats, recruiting minors via Internet services for nefarious purposes, using deceptive means to dupe vulnerable populations, exhibiting misbehaving behaviors such as using profanity or flashing in online video chats, and many others. These issues have a direct negative impact on the social, psychological and in some cases physical well-being of the end users. An important characteristic of these issues is that they fall in a grey legal area, where perpetrators may claim freedom of speech or rights to free expression despite causing harm. The main goal of this inaugural workshop on cybersafety is to bring together the researchers and practitioners from academia, industry, government and research labs working in the area of cybersafety to discuss the unique challenges in addressing various cybersafety issues and to share experiences, solutions, tools, and techniques. The focus is on the detection, prevention and mitigation of various cybersafety issues, as well as education and promoting safe practices.

One problem facing players of competitive games is negative, or toxic, behavior. League of Legends, the largest eSport game, uses a crowdsourcing platform called the Tribunal to judge whether a reported toxic player should be punished or not. The Tribunal is a two stage system requiring reports from those players that directly observe toxic behavior, and human experts that review aggregated reports. While this system has successfully dealt with the vague nature of toxic behavior by majority rules based on many votes, it naturally requires tremendous cost, time, and human efforts. In this paper, we propose a supervised learning approach for predicting crowdsourced decisions on toxic behavior with large-scale labeled data collections; over 10 million user reports involved in 1.46 million toxic players and corresponding crowdsourced decisions. Our result shows good performance in detecting overwhelmingly majority cases and predicting crowdsourced decisions on them. We demonstrate good portability of our classifier across regions. Finally, we estimate the practical implications of our approach, potential cost savings and victim protection.

Online gaming is a multi-billion dollar industry that entertains a large, global population. One unfortunate phenomenon, however, poisons the competition and the fun: cheating. The costs of cheating span from industry-supported expenditures to detect and limit cheating, to victims' monetary losses due to cyber crime. This paper studies cheaters in the Steam Community, an online social network built on top of the world's dominant digital game delivery platform. We collected information about more than 12 million gamers connected in a global social network, of which more than 700 thousand have their profiles flagged as cheaters. We also collected in-game interaction data of over 10 thousand players from a popular multiplayer gaming server. We show that cheaters are well embedded in the social and interaction networks: their network position is largely indistinguishable from that of fair players. We observe that the cheating behavior appears to spread through a social mechanism: the presence and the number of cheater friends of a fair player is correlated with the likelihood of her becoming a cheater in the future. Also, we observe that there is a social penalty involved with being labeled as a cheater: cheaters are likely to switch to more restrictive privacy settings once they are tagged and they lose more friends than fair players. Finally, we observe that the number of cheaters is not correlated with the geographical, real-world population density, or with the local popularity of the Steam Community.