Over the last few years, verifying the credibility of information sources has become a fundamental need to combat disinformation. Here, we present a language-agnostic model designed to assess the reliability of web domains as sources in references across multiple language editions of Wikipedia. Utilizing editing activity data, the model evaluates domain reliability within different articles of varying controversiality, such as Climate Change, COVID-19, History, Media, and Biology topics. Crafting features that express domain usage across articles, the model effectively predicts domain reliability, achieving an F1 Macro score of approximately 0.80 for English and other high-resource languages. For mid-resource languages, we achieve 0.65, while the performance of low-resource languages varies. In all cases, the time the domain remains present in the articles (which we dub as permanence ) is one of the most predictive features. We highlight the challenge of maintaining consistent model performance across languages of varying resource levels and demonstrate that adapting models from higher-resource languages can improve performance. We believe these findings can assist Wikipedia editors in their ongoing efforts to verify citations and may offer useful insights for other user-generated content communities.

The 2022 Russian invasion of Ukraine has seen an intensification in the use of social media by governmental actors in cyber warfare. Wartime communication via memes has been a successful strategy used not only by independent accounts such as @uamemesforces, but also---for the first time in a full-scale interstate war---by official Ukrainian government accounts such as @Ukraine and @DefenceU. We study this prominent example of memetic warfare through the lens of its narratives, and find them to be a key component of success: tweets with a 'victim' narrative garner twice as many retweets. However, malevolent narratives focusing on the enemy resonate more than those about heroism or victims with countries providing more assistance to Ukraine. Our findings present a nuanced examination of Ukraine's influence operations and of the worldwide response to it, thus contributing new insights into the evolution of socio-technical systems in times of war.

Face masks are one of the cheapest and most effective non-pharmaceutical interventions available against airborne diseases such as COVID-19. Unfortunately, they have been met with resistance by a substantial fraction of the populace, especially in the U.S. In this study, we uncover the latent moral values that underpin the response to the mask mandate, and paint them against the country's political backdrop. We monitor the discussion about masks on Twitter, which involves almost 600k users in a time span of 7 months. By using a combination of graph mining, natural language processing, topic modeling, content analysis, and time series analysis, we characterize the responses to the mask mandate of both those in favor and against them. We base our analysis on the theoretical frameworks of Moral Foundation Theory and Hofstede's cultural dimensions. Our results show that, while the anti-mask stance is associated with a conservative political leaning, the moral values expressed by its adherents diverge from the ones typically used by conservatives. In particular, the expected emphasis on the values of authority and purity is accompanied by an atypical dearth of in-group loyalty. We find that after the mandate, both pro- and anti-mask sides decrease their emphasis on care about others, and increase their attention on authority and fairness, further politicizing the issue. In addition, the mask mandate reverses the expression of Individualism-Collectivism between the two sides, with an increase of individualism in the anti-mask narrative, and a decrease in the pro-mask one. We argue that monitoring the dynamics of moral positioning is crucial for designing effective public health campaigns that are sensitive to the underlying values of the target audience.

Unprecedented lockdowns at the start of the COVID-19 pandemic have drastically changed the routines of millions of people, potentially impacting important health-related behaviors. In this study, we use YouTube videos embedded in tweets about diet, exercise and fitness posted before and during COVID-19 to investigate the influence of the pandemic lockdowns on diet and nutrition. In particular, we examine the nutritional profile of the foods mentioned in the transcript, description and title of each video in terms of six macronutrients (protein, energy, fat, sodium, sugar, and saturated fat). These macronutrient values were further linked to demographics to assess if there are specific effects on those potentially having insufficient access to healthy sources of food. Interrupted time series analysis revealed a considerable shift in the aggregated macronutrient scores before and during COVID-19. In particular, whereas areas with lower incomes showed decrease in energy, fat, and saturated fat, those with higher percentage of African Americans showed an elevation in sodium. Word2Vec word similarities and odds ratio analysis suggested a shift from popular diets and lifestyle bloggers before the lockdowns to the interest in a variety of healthy foods, communal sharing of quick and easy recipes, as well as a new emphasis on comfort foods. To the best of our knowledge, this work is novel in terms of linking attention signals in tweets, content of videos, their nutrients profile, and aggregate demographics of the users. The insights made possible by this combination of resources are important for monitoring the secondary health effects of social distancing, and informing social programs designed to alleviate these effects.

Social media has been an important tool in the expansion of the populist message, and it is thought to have contributed to the electoral success of populist parties in the past decade. This study compares how populist parties advertised on Facebook during the 2019 European Parliamentary election. In particular, we examine commonalities and differences in which audiences they reach and on which issues they focus. By using data from Meta (previously Facebook) Ad Library, we analyze 45k ad campaigns by 39 parties, both populist and mainstream, in Germany, United Kingdom, Italy, Spain, and Poland. While populist parties represent just over 20% of the total expenditure on political ads, they account for 40% of the total impressions—most of which from Eurosceptic and far-right parties—thus hinting at a competitive advantage for populist parties on Facebook. We further find that ads posted by populist parties are more likely to reach male audiences, and sometimes much older ones. In terms of issues, populist politicians focus on monetary policy, state bureaucracy and reforms, and security, while the focus on EU and Brexit is on par with non-populist, mainstream parties. However, issue preferences are largely country-specific, thus supporting the view in political science that populism is a “thin ideology”, that does not have a universal, coherent policy agenda. This study illustrates the usefulness of publicly available advertising data for monitoring the populist outreach to, and engagement with, millions of potential voters, while outlining the limitations of currently available data.

Monitoring advertising around controversial issues is an important step in ensuring accountability and transparency of political processes. To that end, we use the Facebook Ads Library to collect 2312 migration-related advertising campaigns in Italy over one year. Our pro- and anti-immigration classifier (F1=0.85) reveals a partisan divide among the major Italian political parties, with anti-immigration ads accounting for nearly 15M impressions. Although composing 47.6% of all migration-related ads, anti-immigration ones receive 65.2% of impressions. We estimate that about two thirds of all captured campaigns use some kind of demographic targeting by location, gender, or age. We find sharp divides by age and gender: for instance, anti-immigration ads from major parties are 17% more likely to be seen by a male user than a female. Unlike pro-migration parties, we find that anti-immigration ones reach a similar demographic to their own voters. However their audience change with topic: an ad from anti-immigration parties is 24% more likely to be seen by a male user when the ad speaks about migration, than if it does not. Furthermore, the viewership of such campaigns tends to follow the volume of mainstream news around immigration, supporting the theory that political advertisers try to “ride the wave” of current news. We conclude with policy implications for political communication: since the Facebook Ads Library does not allow to distinguish between advertisers intentions and algorithmic targeting, we argue that more details should be shared by platforms regarding the targeting configuration of socio-political campaigns.

Vaccine hesitancy is considered as one of the leading causes for the resurgence of vaccine preventable diseases. A non-negligible minority of parents does not fully adhere to the recommended vaccination schedule, leading their children to be partially immunized and at higher risk of contracting vaccine preventable diseases. Here, we leverage more than one million comments of 201,986 users posted from March 2008 to April 2019 on the public online forum BabyCenter US to learn more about such parents. For 32% with geographic location, we find the number of mapped users for each US state resembling the census population distribution with good agreement. We employ Natural Language Processing to identify 6884 and 10,131 users expressing their intention of following the recommended and alternative vaccination schedule, respectively RSUs and ASUs. From the analysis of their activity on the forum we find that ASUs have distinctly different interests and previous experiences with vaccination than RSUs. In particular, ASUs are more likely to follow groups focused on alternative medicine, are two times more likely to have experienced adverse events following immunization, and to mention more serious adverse reactions such as seizure or developmental regression. Content analysis of comments shows that the resources most frequently shared by both groups point to governmental domains (.gov). Finally, network analysis shows that RSUs and ASUs communicate between each other (indicating the absence of echo chambers), however with the latter group being more endogamic and favoring interactions with other ASUs. While our findings are limited to the specific platform analyzed, our approach may provide additional insights for the development of campaigns targeting parents on digital platforms.

In the age of social media, disasters and epidemics usher not only devastation and affliction in the physical world, but also prompt a deluge of information, opinions, prognoses and advice to billions of internet users. The coronavirus epidemic of 2019-2020, or COVID-19, is no exception, with the World Health Organization warning of a possible 'infodemic' of fake news. In this study, we examine the alternative narratives around the coronavirus outbreak through advertisements promoted on Facebook, the largest social media platform in the US. Using the new Facebook Ads Library, we discover advertisers from public health and non-profit sectors, alongside those from news media, politics, and business, incorporating coronavirus into their messaging and agenda. We find the virus used in political attacks, donation solicitations, business promotion, stock market advice, and animal rights campaigning. Among these, we find several instances of possible misinformation, ranging from bioweapons conspiracy theories to unverifiable claims by politicians, to the sale of face masks which may not necessarily protect the wearer. As we make the dataset available to the community, we hope the advertising domain will become an important part of quality control for public health communication and public discourse in general.

In this study, we present a unique demographically representative dataset of 15k US residents that combines technology use logs with surveys on moral views, human values, and emotional contagion. First, we show which values determine the adoption of Health & Fitness mobile applications, finding that users who prioritize the value of purity and de-emphasize values of conformity, hedonism, and security are more likely to use such apps. Further, we achieve a weighted AUROC of .673 in predicting whether individual exercises and find a strong link of exercise to respondent socioeconomic status, as well as the value of loyalty.

In the global move toward urbanization, making sure the people remaining in rural areas are not left behind in terms of development and policy considerations is a priority for governments worldwide. However, it is increasingly challenging to track important statistics concerning this sparse, geographically dispersed population, resulting in a lack of reliable, up-to-date data. In this study, we examine the usefulness of the Facebook Advertising platform, which offers a digital “census” of over two billions of its users, in measuring potential rural-urban inequalities. We focus on Italy, a country where about 30% of the population lives in rural areas. First, we show that the population statistics that Facebook produces suffer from instability across time and incomplete coverage of sparsely populated municipalities. To overcome such limitation, we propose an alternative methodology for estimating Facebook Ads audiences that nearly triples the coverage of the rural municipalities from 19% to 55% and makes feasible fine-grained sub-population analysis. Using official national census data, we evaluate our approach and confirm known significant urban-rural divides in terms of educational attainment and income. Extending the analysis to Facebook-specific user “interests” and behaviors, we provide further insights on the divide, for instance, finding that rural areas show a higher interest in gambling. Notably, we find that the most predictive features of income in rural areas differ from those for urban centres, suggesting researchers need to consider a broader range of attributes when examining rural wellbeing. The findings of this study illustrate the necessity of improving existing tools and methodologies to include under-represented populations in digital demographic studies – the failure to do so could result in misleading observations, conclusions, and most importantly, policies.

Technology has recently been recruited in the war against the ongoing obesity crisis; however, the adoption of Health & Fitness applications for regular exercise is a struggle. In this study, we present a unique demographically representative dataset of 15k US residents that combines technology use logs with surveys on moral views, human values, and emotional contagion. Combining these data, we provide a holistic view of individuals to model their physical exercise behavior. First, we show which values determine the adoption of Health & Fitness mobile applications, finding that users who prioritize the value of purity and de-emphasize values of conformity, hedonism, and security are more likely to use such apps. Further, we achieve a weighted AUROC of .673 in predicting whether individual exercises, and we also show that the application usage data allows for substantially better classification performance (.608) compared to using basic demographics (.513) or internet browsing data (.546). We also find a strong link of exercise to respondent socioeconomic status, as well as the value of happiness. Using these insights, we propose actionable design guidelines for persuasive technologies targeting health behavior modification.

Improving digital literacy and reducing gender inequalities in internet access and use have been recognized as important development targets within the United Nations' Sustainable Development Goals (SDGs). Tracking progress on digital gender inequalities however is challenging due to limited gender-disaggregated data, especially in less developed country contexts and at the subnational level. We show how data obtained from social media advertising APIs can be used to generate a real-time measure of digital gender inequality. Using data from the Facebook (FB) Marketing API, we document gender gaps in FB use across Indian states and examine to what extent state-level development indicators help predict these gender gaps. Higher GDP per capita, literacy and internet penetration are associated with lower gender gaps in FB presence. These state-level development predictors however are only able to explain a small proportion (7.3%) of the overall variation in gender gaps on FB. Employment-related occupational or industry characteristics from within FB emerge as important variables associated with more gender-balanced FB use.

Social media's unfettered access has made it an important venue for health discussion and a resource for patients and their loved ones. However, the quality of information available, as well as the motivations of its posters, has been questioned. This work examines the individuals on social media that are posting questionable health-related information, and in particular promoting cancer treatments which have been shown to be ineffective (making it a kind of misinformation, willful or not). Using a multi-stage user selection process, we study 4,212 Twitter users who have posted about one of 139 such "treatments", and compare them to a baseline of users generally interested in cancer. Considering features capturing user attributes, writing style, and sentiment, we build a classifier which is able to identify users prone to propagate such misinformation at an accuracy of over 90%, providing a potential tool for public health officials to identify such individuals for preventive intervention.

The American Voter - a seminal work in political science - uncovered the multifaceted nature of voting behavior which has been corroborated in electoral research for decades since. In this paper, we leverage The American Voter as an analysis framework in the realm of computational political science, employing the factors of party, personality, and policy to structure the analysis of public discourse on online social media during the 2016 U.S. presidential primaries. Our analysis of 50 million tweets reveals the continuing importance of these three factors; our understanding is also enriched by the application of sentiment analysis techniques. The overwhelmingly negative sentiment of conversations surrounding 10 major presidential candidates reveals more "crosstalk" from Democratic leaning users towards Republican candidates, and less vice-versa. We uncover the lack of moderation as the most discussed personality dimension during this campaign season, as the political field becomes more extreme - Clinton and Rubio are perceived as moderate, while Trump, Sanders, and Cruz are not. While the most discussed issues are foreign policy and immigration, Republicans tweet more about abortion than Democrats who tweet more about gay rights than Republicans. Finally, we illustrate the importance of multifaceted political discourse analysis by applying regression to quantify the impact of party, personality, and policy on national polls.

Food is an integral part of our lives, cultures, and well-being, and is of major interest to public health. The collection of daily nutritional data involves keeping detailed diaries or periodic surveys and is limited in scope and reach. Alternatively, social media is infamous for allowing its users to update the world on the minutiae of their daily lives, including their eating habits. In this work we examine the potential of Twitter to provide insight into US-wide dietary choices by linking the tweeted dining experiences of 210K users to their interests, demographics, and social networks. We validate our approach by relating the caloric values of the foods mentioned in the tweets to the state-wide obesity rates, achieving a Pearson correlation of 0.77 across the 50 US states and the District of Columbia. We then build a model to predict county-wide obesity and diabetes statistics based on a combination of demographic variables and food names mentioned on Twitter. Our results show significant improvement over previous CHI research (Culotta 2014). We further link this data to societal and economic factors, such as education and income, illustrating that areas with higher education levels tweet about food that is significantly less caloric. Finally, we address the somewhat controversial issue of the social nature of obesity (Christakis & Fowler 2007) by inducing two social networks using mentions and reciprocal following relationships.

How are you related to Malawi? Do recent events on the Comoros effect you in any subtle way? Who in your extended social network is in Croatia? We seldom ask ourselves these questions, yet a "long tail" of content beyond our everyday knowledge is waiting to be explored. In this work we propose a recommendation task of creating interest in little-known content by building personalized "bridges" to users. We consider an example task of interesting users in little-known countries, and propose a system which aggregates a user's Twitter profile, network, and tweets to create an interest model, which is then matched to a library of knowledge about the countries. We perform a user study of 69 participants and conduct 11 in-depth interviews in order to evaluate the efficacy of the proposed approach and gather qualitative insight into the effect of multi-faceted use of Twitter on the perception of the bridges. We find the increase in interest concerning little-known content to greatly depend on the pre-existing disposition to it. Additionally, we discover a set of vital properties good bridges must possess, including recency, novelty, emotiveness, and a proper selection of language. Using the proposed approach we aim to harvest the "invisible connections" to make explicit the idea of a "small world" where even a faraway country is more closely connected to you than you might have imagined.

We present DEESSE [1], a tool that enables an exploratory and serendipitous exploration - at entity level, of the content of two different social media: Wikipedia, a user-curated online encyclopedia, and Yahoo Answers, a more unconstrained question/answering forum. DEESSE represents the content of each source as an entity network, which is further enriched with metadata about sentiment, writing quality, and topical category. Given a query entity, entity results are retrieved from the network by employing an algorithm based on a random walk with restart to the query. Following the emerging paradigm of composite retrieval, we organize the results into topically coherent bundles instead of showing them in a simple ranked list.

With the advent of Twitter and other lightweight social-networking services, one might think that it is easier than ever to maintain geographically dispersed, weaker social ties. By contrast, in this study we show that the international Twitter communication landscape is not only still largely predetermined by physical distance, but that it also depends on countries' social, economic, and cultural attributes. We describe a study of an international Twitter mention network of 13 million users across over 100 countries. We show that the Gravity Model, which hypothesizes that the flow between two areas is proportional to their masses (which we approximate using internet penetration) and inversely proportional to the distance between them, is correlated (r=0.68) with the international communication flow. Using this model, along with other social, economic, and cultural variables, we predict the communication volume at Adjusted R2 of $0.80$, with trade, language and racial intolerance especially impacting communication. We discuss the implications of these barriers to communication in the contexts of collaborative work, software design, and recommendation systems.

Every day, thousands of people make donations to humanitarian, political, environmental, and other causes, a large amount of which occur on the Internet. The solicitations for support, the acknowledgment of a donation and the discussion of corresponding issues are often conducted via email, leaving a record of these social phenomena. In this paper, we describe a comprehensive large-scale data-driven study of donation behavior. We analyze a two-month anonymized email log from several perspectives motivated by past studies on charitable giving: (i) demographics, (ii) user interest, (iii) external time-related factors and (iv) social network influence. We show that email captures the demographic peculiarities of different interest groups, for instance, predicting demographic distributions found in US 2012 Presidential Election exit polls. Furthermore, we find that people respond to major national events, as well as to solicitations with special promotions, and that social connections are the most important factor in predicting donation behavior. Specifically, we identify trends not only for individual charities and campaigns, but also for high-level categories such as political campaigns, medical illnesses, and humanitarian relief. Thus, we show the extent to which large-scale email datasets reveal human donation behavior, and explore the limitations of such analysis.

In many cases, when browsing the Web users are searching for specific information or answers to concrete questions. Sometimes, though, users find unexpected, yet interesting and useful results, and are encouraged to explore further. What makes a result serendipitous? We propose to answer this question by exploring the potential of entities extracted from two sources of user-generated content -- Wikipedia, a user-curated online encyclopedia, and Yahoo! Answers, a more unconstrained question/answering forum -- in promoting serendipitous search. In this work, the content of each data source is represented as an entity network, which is further enriched with metadata about sentiment, writing quality, and topical category. We devise an algorithm based on lazy random walk with restart to retrieve entity recommendations from the networks. We show that our method provides novel results from both datasets, compared to standard web search engines. However, unlike previous research, we find that choosing highly emotional entities does not increase user interest for many categories of entities, suggesting a more complex relationship between topic matter and the desirable metadata attributes in serendipitous search.

As mainstream news media and political campaigns start to pay attention to the political discourse online, a systematic analysis of political speech in social media becomes more critical. What exactly do people say on these sites, and how useful is this data in estimating political popularity? In this study we examine Twitter discussions surrounding seven US Republican politicians who were running for the US Presidential nomination in 2011. We show this largely negative rhetoric to be laced with sarcasm and humor and dominated by a small portion of users. Furthermore, we show that using out-of-the-box classification tools results in a poor performance, and instead develop a highly optimized multi-stage approach designed for general-purpose political sentiment classification. Finally, we compare the change in sentiment detected in our dataset before and after 19 Republican debates, concluding that, at least in this case, the Twitter political chatter is not indicative of national political polls.

We present a large-scale study of content reuse networks in a large and highly hierarchical organization. In our study, we combine analysis of a collection of presentations produced by employees with interviews conducted throughout the organization and a survey to study presentation content reuse. Study results show a variety of information needs and behaviors related to content reuse as well as a need for a personalized and socially-integrated networking tool for enabling easy access to previously generated presentation material. In this paper we describe our findings and outline a set of requirements for an effective content reuse facility.

Event tracking is the task of discovering temporal patterns of popular events from text streams. Existing approaches for event tracking have two limitations: scalability and inability to rule out non-relevant portions in text streams. In this study, we propose a novel approach to tackle these limitations. To demonstrate the approach, we track news events across a collection of weblogs spanning a two-month time period.