Proliferation of GPS-enabled mobile devices has brought a plurality of location-aware applications leveraging the location characteristics in the shared content, like photos and check-ins. While these applications provide contextual and relevant information, they also assume geo-tagged contents to be representative of the geo-bounded characteristics of location. In this paper, however, we show that the characteristics geo-tagged contents capture about a location can vary based on the familiarity of user (sharing the content) with the location. Using a large dataset of geo-tagged photos, we learn descriptive spatial photo characteristics and user temporal-location-familiarity to highlight unique characteristics photos capture of location, which vary significantly if taken by locals versus tourists. We then propose a ranking-approach to find most representative photos for a given city. A user-based evaluation shows photos are more diverse and characteristic of location compared to other popular baselines while being representative of how locals and tourists would describe the city.

Recommendation systems rely on various definitions of similarities. These definitions while having numerous design factors in different domains help identify and recommend relevant content. For example, similarity between users, or items, are measured based on, but not limited to, explicit feedback such as ratings, thumbs up; or/and implicit feedback such as clicks, views etc; or/and based on composition of item such as tags, metadata etc. In this paper, we explore a similarity model while very intuitive to find similar items using a very common natural law of attraction between bodies, that is gravitational law. We show how the two attributes, relative mass and distance between the bodies, of gravitation law can be interpreted for an effective personalized recommendations; in both spatial and non-spatial domains. Finally, we illustrate the use of distance and mass in a non-spatial domain and we exhibit the accuracy in recommendations against popular baselines.

Animated GIFs have been around since 1987 and recently gained more popularity on social networking sites. Tumblr, a large social networking and micro blogging platform, is a popular venue to share animated GIFs. Tumblr users follow blogs, generating a feed or posts, and choose to "like' or to "reblog' favored posts. In this paper, we use these actions as signals to analyze the engagement of over 3.9 million posts, and conclude that animated GIFs are significantly more engaging than other kinds of media. We follow this finding with deeper visual analysis of nearly 100k animated GIFs and pair our results with interviews with 13 Tumblr users to find out what makes animated GIFs engaging. We found that the animation, lack of sound, immediacy of consumption, low bandwidth and minimal time demands, the storytelling capabilities and utility for expressing emotions were significant factors in making GIFs the most engaging content on Tumblr. We also found that engaging GIFs contained faces and had higher motion energy, uniformity, resolution and frame rate. Our findings connect to media theories and have implications in design of effective content dashboards, video summarization tools and ranking algorithms to enhance engagement.

There are many cues that can be used to curate media from social networking websites. Beyond metadata, group behavior provide a strong community-based signal for surfacing images, which we show in a user-defined curatorial task. In a departure from mirco-task crowdwork, we observe that the curation inherent in online photo communities guides the discoverability and consumption of the media, which in turn provides a strong signal that can be used in new editorial tasks in a community-supervised manner. We use this approach in tandem with other more conventional multimedia methods (i.e.\ computer vision and contextual metadata) to form a broad multimodal approach to retrieval and recommendation. We present a large-scale system implementation on a real-world curative task for weather images on a web-scale dataset. Finally, we conduct an evaluation of this system using professional editors and find substantial improvements in editorial efficiency.

The quality of user experience online is affected by the relevance and placement of advertisements. We propose a new system for selecting and displaying visual advertisements in image search result sets. Our method compares the visual similarity of candidate ads to the image search results and selects the most visually similar ad to be displayed. The method further selects an appropriate location in the displayed image grid to minimize the perceptual visual differences between the ad and its neighbors. We conduct an experiment with about 900 users and find that our proposed method provides significant improvement in the users' overall satisfaction with the image search experience, without diminishing the users' ability to see the ad or recall the advertised brand.

We present a scalable approach to automatically suggest relevant clothing products, given a single image without metadata. We formulate the problem as cross-scenario retrieval: the query is a real-world image, while the products from online shopping catalogs are usually presented in a clean environment. We divide our approach into two main stages: a) Starting from articulated pose estimation, we segment the person area and cluster promising image regions in order to detect the clothing classes present in the query image. b) We use image retrieval techniques to retrieve visually similar products from each of the detected classes. We achieve clothing detection performance comparable to the state-of-the-art on a very recent annotated dataset, while being more than 50 times faster. Finally, we present a large scale clothing suggestion scenario, where the product database contains over one million products.

New multimedia applications, such as community-created video repositories and tools for synchronous sharing, have revolutionized the ways that media is watched and shared. Effective instrumentation of these applications can enable researchers and system designers to better understand how video is being consumed: that is, how it is being watched, shared, augmented with annotations and otherwise experienced by individuals, by groups, and crowds. In addition to consuming content, however, people also talk about it. Conversational exchanges around video content can take place within applications (e.g., in chat spaces) and/or using separate communication channels (e.g., microblogs). We propose that the actions of these video viewers, with the video object itself and/or with each other around the video object, provide rich data for understanding the semantics and social relevance of various pieces of video content. We illustrate this approach with current research and a novel taxonomy of social multimedia interaction.

A microblogged stream is delivered over time, providing an ongoing commentary of topics, trends, and issues. In this article, we present two methods of finding temporal topics within these Twitter streams. Using a normalized term frequency, we demonstrate how an effective table of contents can be extracted by finding localized "peaky topics". Second, we find "persistent conversations" which have a lower general salience but sustain and persist over the tweet corpus, in effect the whispering conversation that lingers in the background. These methods are demonstrated on a Twitter corpus of 53,000 tweets and a second Twitter corpus of 1.1 million tweets; the methods are generalizable to apply to any normalized scoring metric across a temporal corpus. We propose our method's implications on social media research and systems from a textual and social network analysis perpective.

We propose a model for automatically cropping images based on a diverse set of content and spatial features. We approach this by extracting pixel-level features and aggregating them over possible crop regions. We then learn a regression model to predict the quality of the crop regions, via the degree to which they would overlaps with human-provided crops from these input features. Candidate images can then be cropped based an exhaustive sweep over candidate crop regions, where each region is scored and the highest-scoring region is retained. The system is unique in its ability to incorporate a variety of pixel-level importance cues when arriving at a final cropping recommendation. We test the system on a set of human-cropped images with a large set of features. We find that the system outperforms baseline approaches, particularly when the aspect ratio of the image is very different from the target thumbnail region.

We present a framework for identifying authoritative sources (such as web sites or individual users) that are likely to produce high-quality or interesting images. We construct a directed graph across sources based on the propensity of one source to "cite" the content from another. A graph-centrality measure scores the authority for each source, which could then be applied for retrieval purposes. We apply this method to web image retrieval, where web sites are the sources, and citations are found via copy detection; and on a photo sharing site, where individuals are the sources and citations are users' favorites. We are able to identify primary or influential sources of media while avoiding the computational cost of other approaches.

The future of multimedia community depends on how the community effectively and efficiently recruits, nurtures and retains young talents. Students tends to decide on their majors based on job opportunities and the main question in every student mind while finishing a degree is "which jobs are out there for me?" In this panel, we have gathered people from both academia and industry to discuss job opportunities and career perceptive. The panel will try to basically answer two main questions: (1) Which are the jobs for the fresh graduates of our community? (2) What are the carrier paths in both academia and industry?