The COVID-19 pandemic highlighted the importance of non-traditional data sources, such as mobile phone data, to inform effective public health interventions and monitor adherence to such measures. Previous studies showed how socioeconomic characteristics shaped population response during restrictions and how repeated interventions eroded adherence over time. Less is known about how different population strata changed their response to repeated interventions and how this impacted the resulting mobility network. We study population response during the first and second infection waves of the COVID-19 pandemic in Chile and Spain. Via spatial lag and regression models, we investigate the adherence to mobility interventions at the municipality level in Chile, highlighting the significant role of wealth, labor structure, COVID-19 incidence, and network metrics characterizing business-as-usual municipality connectivity in shaping mobility changes during the two waves. We assess network structural similarities in the two periods by defining mobility hotspots and traveling probabilities in the two countries. As a proof of concept, we simulate and compare outcomes of an epidemic diffusion occurring in the two waves. While differences exist between factors associated with mobility reduction across waves in Chile, underscoring the dynamic nature of population response, our analysis reveals the resilience of the mobility network across the two waves. We test the robustness of our findings recovering similar results for Spain. Finally, epidemic modeling suggests that historical mobility data from past waves can be leveraged to inform future disease spatial invasion models in repeated interventions. This study highlights the value of historical mobile phone data for building pandemic preparedness and lessens the need for real-time data streams for risk assessment and outbreak response. Our work provides valuable insights into the complex interplay of factors driving mobility across repeated interventions, aiding in developing targeted mitigation strategies.

Individuals suffering from Opioid Use Disorder and other socially stigmatized conditions often rely on peer support groups to find comfort and motivation while treating their condition. Many may face barriers in accessing peer support treatment, such as shame and social stigma, seclusion, or mobility restrictions. In this study, we quantitatively characterize the potential of the Reddit community in offering these individuals an online alternative to receiving peer support. By analyzing the social interactions of thousands of users during the start of opioid use recovery, we uncover that a particular Reddit community exhibits many characteristics similar to in-person peer support groups, featuring the exchange of support, trust, status, and similar experiences. We find that the supportive behavior of this community nudges users to change their personal behavior, and promotes abandoning opioid-related communities in favor of recovery-oriented relationships. Finally, we find that recognition, acknowledgment, and knowledge exchange are the most relevant factors in sustained engagement with the recovery community. Given this evidence, we suggest that this online community may constitute a complement or a surrogate to peer support groups when in-person meetings are not desirable or possible. Our work might inspire harm reduction policies and interventions to favor successful rehabilitation and is fundamental for future research about the use of digital media for recovery support.

Being able to automatically recognize notable sites in the physical world using artificial intelligence embedded in mobile devices can pave the way to new forms of urban exploration and open novel channels of interactivity between residents, travellers and cities. Although the development of outdoor recognition systems has been a topic of interest for a while, most works have been limited in geographic coverage due to lack of high quality image data that can be used for training site recognition engines. As a result, prior systems usually lack generality and operate on a limited scope of pre-elected sites. In this work we design a mobile system that can automatically recognise sites of interest and project relevant information to a user that navigates the city. We build a collection of notable sites using Wikipedia and then exploit online services such as Google Images and Flickr to collect large collections of crowd-sourced imagery describing those sites. These images are then used to train minimal deep learning architectures that can be effectively deployed to dedicated applications on mobile devices. By conducting an evaluation and performing a series of online and real-world experiments, we recognise a number of key challenges in deploying site recognition system and highlight the importance of incorporating mobile contextual information to facilitate the visual recognition task. The similarity in the feature maps of objects that undergo identification, the presence of noise in crowd-sourced imagery and arbitrary user induced inputs are among the factors the impede correct classification for deep learning models. We show how curating the training data through the application of a class-specific image de-noising method and the incorporation of information such as user location, orientation and attention patterns can allow for significant improvement in classification accuracy and the election of an end-to-end system that can effectively be used to recognise sites in the wild.

Place ambiance has a huge influence over how we perceive places. Despite its importance, ambiance has been crucially overlooked by urban planners and scientists alike, not least because it is difficult to record and analyze at scale. We explored the possibility of using social media data to reliably map the ambiance of neighborhoods in the entire city of London. To this end, we collected geo-referenced picture tags from Flickr and matched those tags with the words in a newly created ambiance dictionary. In so doing, we made four main contributions: i) map the ambiance of London neighborhoods; ii) ascertain that such a mapping meets residents' expectations, which are derived from a survey we conducted; iii) show that computer vision techniques upon geo-referenced pictures are of predictive power for neighborhood ambiance; and iv) explain each prediction of a neighborhood's ambiance by identifying the picture that best reflects the meaning of that ambiance (e.g., artsy) in that neighborhood (e.g., South Kensington---the richest and most traditional neighborhood---and Shoreditch---among the most progressive and hipster neighborhoods in the city---are both 'artsy' but in very different ways). The combination of the predictive power of mapping ambiance from images and the ability to explain those predictions makes it possible to discover hidden gems across the city at an unprecedented scale.

User participation in online communities is driven by the intertwinement of the social network structure with the crowd-generated content that flows along its links. These aspects are rarely explored jointly and at scale. By looking at how users generate and access pictures of varying beauty on Flickr, we investigate how the production of quality impacts the dynamics of online social systems. We develop a deep learning computer vision model to score images according to their aesthetic value and we validate its output through crowdsourcing. By applying it to over 15 B Flickr photos, we study for the first time how image beauty is distributed over a large-scale social system. Beautiful images are evenly distributed in the network, although only a small core of people get social recognition for them. To study the impact of exposure to quality on user engagement, we set up matching experiments aimed at detecting causality from observational data. Exposure to beauty is double-edged: following people who produce high-quality content increases one's probability of uploading better photos; however, an excessive imbalance between the quality generated by a user and the user's neighbors leads to a decline in engagement. Our analysis has practical implications for improving link recommender systems.

Sarcasm is a peculiar form of sentiment expression, where the surface sentiment differs from the implied sentiment. The detection of sarcasm in social media platforms has been applied in the past mainly to textual utterances where lexical indicators (such as interjections and intensifiers), linguistic markers, and contextual information (such as user profiles, or past conversations) were used to detect the sarcastic tone. However, modern social media platforms allow to create multimodal messages where audiovisual content is integrated with the text, making the analysis of a mode in isolation partial. In our work, we first study the relationship between the textual and visual aspects in multimodal posts from three major social media platforms, i.e., Instagram, Tumblr and Twitter, and we run a crowdsourcing task to quantify the extent to which images are perceived as necessary by human annotators. Moreover, we propose two different computational frameworks to detect sarcasm that integrate the textual and visual modalities. The first approach exploits visual semantics trained on an external dataset, and concatenates the semantics features with state-of-the-art textual features. The second method adapts a visual neural network initialized with parameters trained on ImageNet to multimodal sarcastic posts. Results show the positive effect of combining modalities for the detection of sarcasm across platforms and methods.

The world is a big place. At any given instant something is happening somewhere, but even when nothing is going on people still find ways to generate multimedia data, ranging from social media posts, to photos and videos. A substantial number of these media objects is associated with a location, and in an increasingly mobile and connected world (both in terms of people and devices), this number is only bound to get larger. Yet, in the multimedia literature we observe that many researchers often unwittingly treat the geospatial dimension as if it were a regular feature dimension, despite it requiring special attention. In order to avoid pitfalls and to steer clear of erroneous conclusions, this tutorial aims to teach researchers and students how geotagged multimedia data differs from regular data and to educate them on best practices when dealing with such data. We will cover the lifecycle of geotagged data in multimedia research, where the topics range from how this kind of data is represented, processed, analyzed, and visualized. The tutorial requires both passive and active involvement, where we not only present the material, but the attendees also get the opportunity to interact with it using a variety of open source data and tools that we have prepared using a virtual machine.

User interfaces for web image search engine results differ significantly from interfaces for traditional (text) web search results, supporting a richer interaction. In particular, users can see an enlarged image preview by hovering over a result image, and an `image preview' page allows users to browse further enlarged versions of the results, and to click-through to the referral page where the image is embedded. No existing work investigates the utility of these interactions as implicit relevance feedback for improving search ranking, beyond using clicks on images displayed in the search results page. In this paper we propose a number of implicit relevance feedback features based on these additional interactions: hover-through rate, 'converted-hover' rate, referral page click through, and a number of dwell time features. Also, since images are never self-contained, but always embedded in a referral page, we posit that clicks on other images that are embedded on the same referral webpage as a given image can carry useful relevance information about that image. We also posit that query-independent versions of implicit feedback features, while not expected to capture topical relevance, will carry feedback about the quality or attractiveness of images, an important dimension of relevance for web image search. In an extensive set of ranking experiments in a learning to rank framework, using a large annotated corpus, the proposed features give statistically significant gains of over 2% compared to a state of the art baseline that uses standard click features.

In this study, we analyze user image search behavior from a large-scale Yahoo! Image Search query log, based on the hypothesis that behavior is dependent on query type. We categorize queries using two orthogonal taxonomies (subject-based and facet-based) and identify important query types at the intersection of these taxonomies. We study user search behavior on a large-scale set of search sessions for each query type, examining characteristics of sessions, query reformulation patterns, click patterns, and page view patterns. We identify important behavioral differences across query types, in particular showing that some query types are more exploratory, while others correspond to focused search. We also supplement our study with a survey to link the behavioral differences to users' intent. Our findings shed light on the importance of considering query categories to better understand user behavior on image search platforms.

Walkability has many health, environmental, and economic benefits. That is why web and mobile services have been offering ways of computing walkability scores of individual street segments. Those scores are generally computed from survey data and manual counting (of even trees). However, that is costly, owing to the high time, effort, and financial costs. To partly automate the computation of those scores, we explore the possibility of using the social media data of Flickr and Foursquare to automatically identify safe and walkable streets. We find that unsafe streets tend to be photographed during the day, while walkable streets are tagged with walkability-related keywords. These results open up practical opportunities (for, e.g., room booking services, urban route recommenders, and real-estate sites) and have theoretical implications for researchers who might resort to the use social media data to tackle previously unanswered questions in the area of walkability.

The notion of creativity, as opposed to related concepts such as beauty or interestingness, has not been studied from the perspective of automatic analysis of multimedia content. Meanwhile, short online videos shared on social media platforms, or micro-videos, have arisen as a new medium for creative expression. In this paper we study creative micro-videos in an effort to understand the features that make a video creative, and to address the problem of automatic detection of creative content. Defining creative videos as those that are novel and have aesthetic value, we conduct a crowdsourcing experiment to create a dataset of over 3, 800 micro-videos labelled as creative and non-creative. We propose a set of computational features that we map to the components of our definition of creativity, and conduct an analysis to determine which of these features correlate most with creative video. Finally, we evaluate a supervised approach to automatically detect creative video, with promising results, showing that it is necessary to model both aesthetic value and novelty to achieve optimal classification accuracy.

Online social networks and mash-up services create opportunities to connect different web services otherwise isolated. Specifically in the case of news, users are very much exposed to news articles while performing other activities, such as social networking or web searching. Browsing behavior aimed at the consumption of news, especially in relation to the visits coming from other domains, has been mainly overlooked in previous work. To address that, we build a BrowseGraph out of the collective browsing traces extracted from a large viewlog of Yahoo News (0.5B entries), and we define the ReferrerGraph as its subgraph induced by the sessions with the same referrer domain. The structural and temporal properties of the graph show that browsing behavior in news is highly dependent on the referrer URL of the session, in terms of type of content consumed and time of consumption. We build on this observation and propose a news recommender that addresses the cold-start problem: given a user landing on a page of the site for the first time, we aim to predict the page she will visit next. We compare 24 flavors of recommenders belonging to the families of content-based, popularity-based, and browsing-based models. We show that the browsing-based recommender that takes into account the referrer URL is the best performing, achieving a prediction accuracy of 48% in conditions of heavy data sparsity.

Every day millions of users are connected through online social networks, generating a rich trove of data that allows us to study the mechanisms behind human interactions. Triadic closure has been treated as the major mechanism for creating social links: if Alice follows Bob and Bob follows Charlie, Alice will follow Charlie. Here we present an analysis of longitudinal micro-blogging data, revealing a more nuanced view of the strategies employed by users when expanding their social circles. While the network structure affects the spread of information among users, the network is in turn shaped by this communication activity. This suggests a link creation mechanism whereby Alice is more likely to follow Charlie after seeing many messages by Charlie. We characterize users with a set of parameters associated with different link creation strategies, estimated by a Maximum-Likelihood approach. Triadic closure does have a strong effect on link formation, but shortcuts based on traffic are another key factor in interpreting network evolution. However, individual strategies for following other users are highly heterogeneous. Link creation behaviors can be summarized by classifying users in different categories with distinct structural and behavioral characteristics. Users who are popular, active, and influential tend to create traffic-based shortcuts, making the information diffusion process more efficient in the network.

Social media have attracted considerable attention because their open-ended nature allows users to create lightweight semantic scaffolding to organize and share content. To date, the interplay of the social and topical components of social media has been only partially explored. Here, we study the presence of homophily in three systems that combine tagging social media with online social networks. We find a substantial level of topical similarity among users who are close to each other in the social network. We introduce a null model that preserves user activity while removing local correlations, allowing us to disentangle the actual local similarity between users from statistical effects due to the assortative mixing of user activity and centrality in the social network. This analysis suggests that users with similar interests are more likely to be friends, and therefore topical similarity measures among users based solely on their annotation metadata should be predictive of social links. We test this hypothesis on several datasets, confirming that social networks constructed from topical similarity capture actual friendship accurately. When combined with topological features, topical similarity achieves a link prediction accuracy of about 92%.

We introduce the Chain Model for object association games, and two social tagging games based on this model. GiveALink Slider and Great Minds Think Alike harness human power to generate large streams of high-quality social tagging data. Such social annotations are utilized to help people organize Web resources and infer semantic relationship, which in turn can enhance Web applications such as search, recommendation, navigation, and categorization. The two games leverage several design features as well as external social media resources to create entertaining incentives for the players to generate reliable annotation data.

This article examines the use of two kinds of context to improve the results of content-based music taggers: the relationships between tags and between the clips of songs that are tagged. We show that users agree more on tags applied to clips temporally “closer” to one another; that conditional restricted Boltzmann machine models of tags can more accurately predict related tags when they take context into account; and that when training data is “smoothed” using context, support vector machines can better rank these clips according to the original, unsmoothed tags and do this more accurately than three standard multi-label classifiers.

Web 2.0 applications have attracted a considerable amount of attention because their open-ended nature allows users to create lightweight semantic scaffolding to organize and share content. To date, the interplay of the social and semantic components of social media has been only partially explored. Here we focus on Flickr and Last.fm, two social media systems in which we can relate the tagging activity of the users with an explicit representation of their social network. We show that a substantial level of local lexical and topical alignment is observable among users who lie close to each other in the social network. We introduce a null model that preserves user activity while removing local correlations, allowing us to disentangle the actual local alignment between users from statistical effects due to the assortative mixing of user activity and centrality in the social network. This analysis suggests that users with similar topical interests are more likely to be friends, and therefore semantic similarity measures among users based solely on their annotation metadata should be predictive of social links. We test this hypothesis on the Last.fm data set, confirming that the social network constructed from semantic similarity captures actual friendship more accurately than Last.fm's suggestions based on listening patterns.

Network analysis has proved to be very useful in many social and natural sciences, and in particular Small World topologies have been exploited in many application fields. In this article, we focus on P2P file sharing applications, where spontaneous communities of users are studied and analyzed. We define a family of structures that we call “Affinity Networks” (or even Graphs) that show self-organized interest-based clusters. Empirical evidence proves that affinity networks are small worlds and shows scale-free features. The relevance of this finding is augmented with the introduction of a proactive recommendation scheme, namely DeHinter , that exploits this natural feature. The intuition behind this scheme is that a user would trust her network of “elective affinities” more than anonymous and generic suggestions made by impersonal entities. The accuracy of the recommendation is evaluated by way of a 10-fold cross validation, and a prototype has been implemented for further feedbacks from the users.

Peer-to-peer (P2P) systems are widely used for sharing digital items without structured metadata and in absence of any kind of digital rights management applied to the distributed contents. In this paper we propose the implementation of a prototype application that makes use of a structured P2P system enabling the indexing of complex metadata, used to express digital rights. In this way the media contents are exchanged and played according to the expressed grants. The creation and the consumption of the shared contents can be performed through any MPEG-21 REL compliant software and the application allows indexing and search for both governed and ungoverned contents. The information about the license can be included in the queries and the P2P network can be used to share governed contents (both free and with fee) in a legitimate way. In particular the proposed approach represents a suitable solution for indexing and querying rights complex structures on DHT based networks.

The lack of a trusted authority, responsible for peers' identity verification or for authentication purposes, makes actual P2P systems extremely vulnerable to a large spectrum of attacks. The main purpose of this paper is to present Likir (layered identity-based Kademlia-like infrastructure), a framework that includes an identity-based scheme and a secure communication protocol, built on top of Kademlia, that may provide an effective defense against well known attacks. This will be accomplished with the adoption of a certification service, with the use of an authentication protocol between nodes and with the introduction of credentials to make non-repudiable the ownership of the contents and messages inserted in the DHT. For sake of interoperability with other social networking services, Likir enables identity management under the Identity 2.0 framework. Under this perspective, the IBS (identity-based signature) scheme is taken into consideration and analyzed as well.

We focus on collaborative filtering dealing with self-organizing communities, host mobility, wireless access, and ad-hoc communications. In such a domain, knowledge representation and users profiling can be hard; remote servers can be often unreachable due to client mobility; and feedback ratings collected during random connections to other users' ad-hoc devices can be useless, because of natural differences between human beings. Our approach is based on so called Affinity Networks, and on a novel system, called MobHinter, that epidemically spreads recommendations through spontaneous similarities between users. Main results of our study are two fold: firstly, we show how to reach comparable recommendation accuracies in the mobile domain as well as in a complete knowledge scenario; secondly, we propose epidemic collaborative strategies that can reduce rapidly and realistically the cold start problem.

We present WALTy (Web application load-based testing tool), a set of tools that allows the performance analysis of Web applications by means of a scalable what-if analysis on the test bed. The proposed approach is based on a workload characterization generated from information extracted from log files. The workload is generated by using of customer behavior model graphs (CBMG), that are derived by extracting information from the Web application log files. In this manner the synthetic workload used to evaluate the Web application under test is representative of the real traffic that the Web application has to serve. One of the most common critics to this approach is that synthetic workload produced by Web stressing tools is far from being realistic. The use of the CBMGs might be useful to overcome this critic.