Legislation in the USA will soon require that electric vehicles (EVs) display battery degradation in the instrument cluster as "state of health" (SOH), the percentage of the battery's original capacity.However, the extent to which consumers understand SOH degradation patterns is not known.In an initial study with vehicle owners, we find preliminary evidence for a 'SOH illusion, ' wherein people expect linear rates of EV battery degradation over time even though batteries degrade non-linearly.Additionally, a third of participants incorrectly confused SOH with a battery's remaining usable life, demonstrating some misunderstanding of SOH among vehicle owners.In a follow-up study we find that framing SOH information with different chart types and legends reduces linear degradation assumptions and aligns people's expectations.We discuss implications for the design of SOH representations in user interfaces that vehicle UI designers could employ to promote better EV battery understanding.

The Wizard of Oz (WoZ) method is a widely adopted research approach where a human Wizard “role-plays” a not readily available technology and interacts with participants to elicit user behaviors and probe the design space. With the growing ability for modern large language models (LLMs) to role-play, one can apply LLMs as Wizards in WoZ experiments with better scalability and lower cost than the traditional approach. However, methodological guidance on responsibly applying LLMs in WoZ experiments and a systematic evaluation of LLMs’ role-playing ability are lacking. Through two LLM-powered WoZ studies, we take the first step towards identifying an experiment lifecycle for researchers to safely integrate LLMs into WoZ experiments and interpret data generated from settings that involve Wizards role-played by LLMs. We also contribute a heuristic-based evaluation framework that allows the estimation of LLMs’ role-playing ability in WoZ experiments and reveals LLMs’ behavior patterns at scale.

With the rising popularity of electrified vehicles, emphasis has been placed on encouraging charging with renewable energy and maximizing battery longevity to improve vehicle sustainability. Many mobile applications offer tools to suggest charging times with more sustainable renewable energy and charging strategies that preserve battery health. However, these options often result in longer, less convenient charging times for drivers. Here we conducted three charging scenario studies to identify factors that influence willingness to wait for sustainable charging. Participants selected between faster but less sustainable charging options and slower charging options that either reduce charging emissions or improve battery longevity. We find people’s willingness to wait for green energy is influenced by situational factors; further we find that information and battery longevity interventions can increase willingness to wait for sustainable charging. Finally, we provide design recommendations to promote sustainably in charging behaviors.

Electrification is an important first step toward reducing the greenhouse emissions of passenger vehicles. However, how drivers drive, charge, and operate their electrified vehicles can have a large impact on their emissions, particularly for Plug-in Hybrid Electric vehicles (PHEVs) that combine all-electric driving with an internal combustion engine. In this paper, we investigate how and why drivers use their PHEVs and uncover design opportunities for interfaces that can support the efficient use of PHEVs. We used a mixed-method approach combining quantitative, qualitative, and concept elicitation methods with PHEV owners in the US. While past findings indicate that PHEV drivers are not motivated to charge regularly, our work contradicts this with evidence of (1) regular charging with home infrastructure, (2) high cost sensitivity, and (3) preference for driving in all-electric mode. Our results indicate that the most critical problem is inadequate user support for navigating poor charging infrastructure.

From ride-hailing to car rentals, consumers are often presented with eco-friendly options. Beyond highlighting a “green” vehicle and CO2 emissions, CO2 equivalencies have been designed to provide understandable amounts; we ask which equivalencies will lead to eco-friendly decisions. We conducted five ride-hailing scenario surveys where participants picked between regular and eco-friendly options, testing equivalencies, social features, and valence-based interventions. Further, we tested a car-rental embodiment to gauge how an individual (needing a car for several days) might behave versus the immediate ride-hailing context. We find that participants are more likely to choose green rides when presented with additional information about emissions; CO2 by weight was found to be the most effective. Further, we found that information framing—be it individual or collective footprint, positive or negative valence—had an impact on participants’ choices. Finally, we discuss how our findings inform the design of effective interventions for reducing car-based carbon-emissions.

Behaviour in virtual environments might be informed by our experiences in physical environments, but virtual environments are not constrained by the same physical, perceptual, or social cues. Instead of replicating the properties of physical spaces, one can create virtual experiences that diverge from reality by dynamically manipulating environmental, aural, and social properties. This paper explores digital proxemics, which describe how we use space in virtual environments and how the presence of others influences our behaviours, interactions, and movements. First, we frame the open challenges of digital proxemics in terms of activity, social signals, audio design, and environment. We explore a subset of these challenges through an evaluation that compares two audio designs and two displays with different social signal affordances: head-mounted display (HMD) versus desktop PC. We use quantitative methods using instrumented tracking to analyse behaviour, demonstrating how personal space, proximity, and attention compare between desktop PC and HMDs.

Virtual environments (VEs) can create collaborative and social spaces, which are increasingly important in the face of remote work and travel reduction. Recent advances, such as more open and widely available platforms, create new possibilities to observe and analyse interaction in VEs. Using a custom instrumented build of Mozilla Hubs to measure position and orientation, we conducted an academic workshop to facilitate a range of typical workshop activities. We analysed social interactions during a keynote, small group breakouts, and informal networking/hallway conversations. Our mixed-methods approach combined environment logging, observations, and semi-structured interviews. The results demonstrate how small and large spaces influenced group formation, shared attention, and personal space, where smaller rooms facilitated more cohesive groups while larger rooms made small group formation challenging but personal space more flexible. Beyond our findings, we show how the combination of data and insights can fuel collaborative spaces’ design and deliver more effective virtual workshops.

Augmented reality (AR) on smartphone devices allows people to interact with virtually placed objects anchored in the real world through the device’s viewport. Typically, smartphone AR interactions work with the device’s 2D touchscreen disconnected from the modality and depth of the virtual objects. In this paper, we studied 15 participants’ preferences, performances, and cognitive loads on a set of common tasks performed on smartphones at two interaction depths (close-range and distant) with two touchless modalities (hand tracking and screen dwell). We find that distant AR interactions, strongly preferred by the participants, were significantly faster and took less cognitive effort. We observed that within both interaction depths, modalities performed equally. When designing touchless modalities on smartphones, we suggest using distant interactions when overall performance is the top priority, otherwise, using hand tracking or screen dwell as back-ups for each other can be equally effective.

McCarthy (1998) defined Artificial Intelligence (AI) as both “the science and engineering of in- telligent machines, especially computer programs” and the “computational part of the ability to achi...

Multimedia research has now moved beyond laboratory experiments and is rapidly being deployed in real-life applications including advertisements, social interaction, search, security, automated driving, and healthcare. Hence, the developed algorithms now have a direct impact on the individuals using the abovementioned services and the society as a whole. While there is a huge potential to benefit the society using such technologies, there is also an urgent need to identify the checks and balances to ensure that the impact of such technologies is ethical and positive. This panel will bring together an array of experts who have experience collecting large-scale datasets, building multimedia algorithms, and deploying them in practical applications, as well as, a lawyer whose eyes have been on the fundamental rights at stake. They will lead a discussion on the ethics and lawfulness of dataset creation, licensing, privacy of individuals represented in the datasets, algorithmic transparency, algorithmic bias, explainability, and the implications of application deployment. Through an interactive process engaging the audience, the panel hopes to: increase the awareness of such concepts in the multimedia research community; initiate a discussion on community guidelines all for setting the future direction of conducting multimedia research in a lawful and ethical manner.

It is our great pleasure to welcome you to this issue of the Proceedings of the ACM on Human- Computer Interaction, on the contributions of the research community Computer-Supported Cooperative Work and Social Computing (CSCW). This issue contains a carefully selected set of papers, accepted through our review process from among the 658 world-wide articles submitted by the Spring 2019 deadline. After the first round of peer review, 325 (49.4%) papers were invited to the Revise and Resubmit phase. After receiving the revised submissions, the external reviewers and the program committee reviewed all second round contributions. Finally, the program committee came together for a three-day online editorial committee meeting, held to allow for collective deliberation. Ultimately, 205 papers (31.2%) were accepted.

Massive Open Online Course (MOOC) platforms have scaled online education to unprecedented enrollments, but remain limited by their rigid, predetermined curricula. To overcome this limitation, this paper contributes a visual recommender system called MOOCex. The system recommends lecture videos across different courses by considering both video contents and sequential inter-topic relationships mined from course syllabi; and more importantly, it allows for interactive visual exploration of the semantic space of recommendations within a learner's current context. When compared to traditional methods (e.g., content-based recommendation and ranked list representations), MOOCex suggests videos from more diverse perspectives and helps learners make better video playback decisions. Further, feedback from MOOC learners and instructors indicates that the system enhances both learning and teaching effectiveness.

Massive Open Online Course (MOOC) platforms have scaled online education to unprecedented enrollments, but remain limited by their predetermined curricula. Increasingly, professionals consume this content to augment or update specific skills rather than complete degree or certification programs. To better address the needs of this emergent user population, we describe a visual recommender system called MOOCex. The system recommends lecture videos across multiple courses and content platforms to provide a choice of perspectives on topics of interest. The recommendation engine considers both video content and sequential inter-topic relationships mined from course syllabi. Furthermore, it allows for interactive visual exploration of the semantic space of recommendations within a learner's current context.

Social media sites are challenged by both the scale and variety of deviant behavior online. While algorithms can detect spam and obscenity, behaviors that break community guidelines on some sites are difficult because they have multimodal subtleties (images and/or text). Identifying these posts is often regulated to a few moderators. In this paper, we develop a deep learning classifier that jointly models textual and visual characteristics of pro-eating disorder content that violates community guidelines. Using a million Tumblr photo posts, our classifier discovers deviant content efficiently while also maintaining high recall (85%). Our approach uses human sensitivity throughout to guide the creation, curation, and understanding of this approach to challenging, deviant content. We discuss how automation might impact community moderation, and the ethical and social obligations of this area.

The CWI-ADE2016 Dataset is a collection of more than 40 million Bluetooth Low Energy (BLE) packets and of 14 million accelerometer and temperature samples generated by wristbands that people wore in a nightclub. The data was gathered during Amsterdam Dance Event 2016 in an exclusive club experience curated around human senses, which leveraged technology as a bridge between the club and the guests. Each guest was handed a custom-made wristband with a BLE-enabled device that broadcast movement, temperature and other sensor readings. A network of Raspberry Pi receivers deployed for the occasion captured broadcast packets from wristbands and any other BLE device in the environment. This data provides a full picture of the performance of the real life deployment of a sensing infrastructure and gives insights to designing sensing platforms, understanding networks and crowds behaviour or studying opportunistic sensing. This paper describes an analysis of this dataset and some examples of usage.

Proliferation of GPS-enabled mobile devices has brought a plurality of location-aware applications leveraging the location characteristics in the shared content, like photos and check-ins. While these applications provide contextual and relevant information, they also assume geo-tagged contents to be representative of the geo-bounded characteristics of location. In this paper, however, we show that the characteristics geo-tagged contents capture about a location can vary based on the familiarity of user (sharing the content) with the location. Using a large dataset of geo-tagged photos, we learn descriptive spatial photo characteristics and user temporal-location-familiarity to highlight unique characteristics photos capture of location, which vary significantly if taken by locals versus tourists. We then propose a ranking-approach to find most representative photos for a given city. A user-based evaluation shows photos are more diverse and characteristic of location compared to other popular baselines while being representative of how locals and tourists would describe the city.

Recommendation systems rely on various definitions of similarities. These definitions while having numerous design factors in different domains help identify and recommend relevant content. For example, similarity between users, or items, are measured based on, but not limited to, explicit feedback such as ratings, thumbs up; or/and implicit feedback such as clicks, views etc; or/and based on composition of item such as tags, metadata etc. In this paper, we explore a similarity model while very intuitive to find similar items using a very common natural law of attraction between bodies, that is gravitational law. We show how the two attributes, relative mass and distance between the bodies, of gravitation law can be interpreted for an effective personalized recommendations; in both spatial and non-spatial domains. Finally, we illustrate the use of distance and mass in a non-spatial domain and we exhibit the accuracy in recommendations against popular baselines.

Despite progress in perceptual tasks such as image classification, computers still perform poorly on cognitive tasks such as image description and question answering. Cognition is core to tasks that involve not just recognizing, but reasoning about our visual world. However, models used to tackle the rich content in images for cognitive tasks are still being trained using the same datasets designed for perceptual tasks. To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in an image. When asked “What vehicle is the person riding?”, computers will need to identify the objects in an image as well as the relationships riding(man, carriage) and pulling(horse, carriage) to answer correctly that “the person is riding a horse-drawn carriage.” In this paper, we present the Visual Genome dataset to enable the modeling of such relationships. We collect dense annotations of objects, attributes, and relationships within each image to learn these models. Specifically, our dataset contains over 108K images where each image has an average of $$35$$ objects, $$26$$ attributes, and $$21$$ pairwise relationships between objects. We canonicalize the objects, attributes, relationships, and noun phrases in region descriptions and questions answer pairs to WordNet synsets. Together, these annotations represent the densest and largest dataset of image descriptions, objects, attributes, relationships, and question answer pairs.

People taking photographs with their tablets is an increasingly common sight. While the design of hardware is larger and more cumbersome than the smaller and more often carried cameraphones, the underlying photographic software is essentially the same. At the same time, cameraphone hardware usually contains objectively better quality cameras with more megapixels, better lenses, and faster image processing. So why do people choose one over the other? In this paper we explore the experience of tablet photography and aim to compare it with camera phone photography. In a mixed-methods study, we use quantitative and qualitative data to explore perceptions of photo quality, factors influencing photo quality, and the affordances of tablet computers that lead users to prefer them over camera phones. Our results indicate that participants take photos with their camera phone for the convenience of the small device form factor, but prefer taking photos with their larger tablet for photos that require higher definition. We discuss the design implications of this research for the design of mobile cameras.

Online photo collections are often presented along with numeric data, such as views, likes, or comments. Treemaps are ideal for visualizing such collections, as they present numeric values using rectangular blocks, within which photos can be presented. Despite abundant research showing that photo treemaps and similar space-filling approaches are useful and appealing for users, understanding of the ideal parameters for constructing these representations remains limited. To address this, we contribute a series of experiments targeted at identifying design parameters for building photo treemaps. Our first study explores the number of photos presented at each level in the treemap, finding that using fewer photos makes visualizations more searchable and preferable to users. A second study examines the tradeoff between sizing photos according to the numeric values and presenting them in a more familiar layout with fixed aspect ratios. These results inform a third study, in which a prototype was used to probe the preferences of active Flickr users for using treemaps for navigating photo collections.

Animated GIFs have been around since 1987 and recently gained more popularity on social networking sites. Tumblr, a large social networking and micro blogging platform, is a popular venue to share animated GIFs. Tumblr users follow blogs, generating a feed or posts, and choose to "like' or to "reblog' favored posts. In this paper, we use these actions as signals to analyze the engagement of over 3.9 million posts, and conclude that animated GIFs are significantly more engaging than other kinds of media. We follow this finding with deeper visual analysis of nearly 100k animated GIFs and pair our results with interviews with 13 Tumblr users to find out what makes animated GIFs engaging. We found that the animation, lack of sound, immediacy of consumption, low bandwidth and minimal time demands, the storytelling capabilities and utility for expressing emotions were significant factors in making GIFs the most engaging content on Tumblr. We also found that engaging GIFs contained faces and had higher motion energy, uniformity, resolution and frame rate. Our findings connect to media theories and have implications in design of effective content dashboards, video summarization tools and ranking algorithms to enhance engagement.

Temporal terms, such as 'winter', 'Christmas', or 'January' are often used in search queries for personal images. But how do people's memories and perceptions of time match with the actual dates when their images were captured? We compared the temporal terms that 74 Flickr users used to search their own photo collections, and compared them to the date captured data in the target image. We also conducted a larger study across several billion images, comparing user-applied tags for holidays and seasons to the dates the images were captured. We demonstrate that various query terms and tags can be in conflict with the actual dates photos were taken for specific types of temporal terms up to 40% of the time. We will conclude by highlighting implications for search systems where users are querying for personal content by date.

Participants in social media systems must balance many considerations when choosing what to share and with whom. Sharing with others invites certain risks, as well as potential benefits; achieving the right balance is even more critical when sharing photos, which can be particularly engaging, but potentially compromising. In this paper, we examine photo-sharing decisions as an interaction between high-level user preferences and specific features of the images being shared. Our analysis combines insights from a 96-user survey with metadata from 10.4M photos to develop a model integrating these perspectives to predict permissions settings for uploaded photos. We discuss implications, including how such a model can be applied to provide online sharing experiences that are more safe, more scalable, and more satisfying.

Microtask crowdsourcing has enabled dataset advances in social science and machine learning, but existing crowdsourcing schemes are too expensive to scale up with the expanding volume of data. To scale and widen the applicability of crowdsourcing, we present a technique that produces extremely rapid judgments for binary and categorical labels. Rather than punishing all errors, which causes workers to proceed slowly and deliberately, our technique speeds up workers' judgments to the point where errors are acceptable and even expected. We demonstrate that it is possible to rectify these errors by randomizing task order and modeling response latency. We evaluate our technique on a breadth of common labeling tasks such as image verification, word similarity, sentiment analysis and topic classification. Where prior work typically achieves a 0.25x to 1x speedup over fixed majority vote, our approach often achieves an order of magnitude (10x) speedup.

There are many cues that can be used to curate media from social networking websites. Beyond metadata, group behavior provide a strong community-based signal for surfacing images, which we show in a user-defined curatorial task. In a departure from mirco-task crowdwork, we observe that the curation inherent in online photo communities guides the discoverability and consumption of the media, which in turn provides a strong signal that can be used in new editorial tasks in a community-supervised manner. We use this approach in tandem with other more conventional multimedia methods (i.e.\ computer vision and contextual metadata) to form a broad multimodal approach to retrieval and recommendation. We present a large-scale system implementation on a real-world curative task for weather images on a web-scale dataset. Finally, we conduct an evaluation of this system using professional editors and find substantial improvements in editorial efficiency.

The quality of user experience online is affected by the relevance and placement of advertisements. We propose a new system for selecting and displaying visual advertisements in image search result sets. Our method compares the visual similarity of candidate ads to the image search results and selects the most visually similar ad to be displayed. The method further selects an appropriate location in the displayed image grid to minimize the perceptual visual differences between the ad and its neighbors. We conduct an experiment with about 900 users and find that our proposed method provides significant improvement in the users' overall satisfaction with the image search experience, without diminishing the users' ability to see the ad or recall the advertised brand.

Points of interest are an important requirement for location-based services, yet they are editorially curated and maintained, either professionally or through community. Beyond the laborious manual annotation task, further complications arise as points of interest may appear, relocate, or disappear over time, and may be relevant only to specific communities. To assist, complement, or even replace manual annotation, we propose a novel method for the automatic localization of points of interest depicted in photos taken by people across the world. Our technique exploits the geographic coordinates and the compass direction supplied by modern cameras, while accounting for possible measurement errors due to the variability in accuracy of the sensors that produced them. We statistically demonstrate that our method significantly outperforms techniques from the research literature on the task of estimating the geographic coordinates and geographic footprints of points of interest in various cities, even when photos are involved in the estimation process that do not show the point of interest at all.

Increasingly online reviews are relied upon to make choices about the purchases and services we use daily. Businesses, on the other hand, depend on online review sites to find new customers and understand people's perception of them. In order for an online review community to be effective to both users and businesses, it is important to understand what constitutes a high quality review as perceived by people, and how to maximize quality of reviews in the community. In this paper, we study Yelp to answer these questions. We analyze about 230,000 reviews and member interaction ("votes") with these reviews. We find that active and regular members are the highest contributors to good quality reviews and longer reviews have higher chances of being popular in the community. We find that reviews voted "useful" tend to be the early ones for a specific business. Our findings have implications on enabling high quality member contributions and community effectiveness. We discuss the implications to design of social systems with diverse feedback signals.

This paper develops a novel framework for semantic image retrieval based on the notion of a scene graph. Our scene graphs represent objects (“man”, “boat”), attributes of objects (“boat is white”) and relationships between objects (“man standing on boat”). We use these scene graphs as queries to retrieve semantically related images. To this end, we design a conditional random field model that reasons about possible groundings of scene graphs to test images. The likelihoods of these groundings are used as ranking scores for retrieval. We introduce a novel dataset of 5,000 human-generated scene graphs grounded to images and use this dataset to evaluate our method for image retrieval. In particular, we evaluate retrieval using full scene graphs and small scene subgraphs, and show that our method outperforms retrieval methods that use only objects or low-level image features. In addition, we show that our full model can be used to improve object localization compared to baseline methods.

With the use of several tablet devices and a shared large display, CozyMaps is a multi-display system that supports real-time collocated collaboration on a shared map. This paper builds on existing works and introduces rich user interactions by proposing awareness, notification, and view sharing techniques, to enable seamless information sharing and integration in map-based applications. Based on our exploratory study, we demonstrated that participants are satisfied with these new proposed interactions. We found that view sharing techniques should be location-focused rather than user-focused. Our results provide implications for the design of interactive techniques in collaborative multi-display map systems.

This ACM MM panel aims to redefine the state of research between Academia and Industry.

The Multimedia COMMONS workshop laid the groundwork for developing a research community around the Multimedia Genome Project (MMGP), an initiative initially focused on annotation of---and research using---the 99.2 million images and nearly 800,000 videos in the Yahoo Flickr Creative Commons 100 Million dataset (YFCC100M). Current and potential users of the YFCC100M presented new research and systems that used this unprecedentedly large, unprecedentedly open-source dataset; discussed ideas for future data challenges and new benchmarking tasks that would not previously have been possible; and suggested priorities and plans for annotation and distribution based on community needs and interests.

The Arts and Digital Culture program has offered a high quality forum for the presentation of interactive and arts-based multimedia applications at the annual ACM Multimedia conference for over a decade. This tutorial will explore the evolution of this program as a guide to new authors considering future participation in this program. By surveying both past technical and past exhibited contributions, this tutorial will offer guidance to artists, researchers and practitioners on success at this multifaceted, interdisciplinary forum at ACM Multimedia.

Geotagged data can be used to describe regions in the world and discover local themes. However, not all data produced within a region is necessarily specifically descriptive of that area. To surface the content that is characteristic for a region, we present the geographical hierarchy model (GHM), a probabilistic model based on the assumption that data observed in a region is a random mixture of content that pertains to different levels of a hierarchy. We apply the GHM to a dataset of 8 million Flickr photos in order to discriminate between content (i.e. tags) that specifically characterizes a region (e.g. neighborhood) and content that characterizes surrounding areas or more general themes. Knowledge of the discriminative and non-discriminative terms used throughout the hierarchy enables us to quantify the uniqueness of a given region and to compare similar but distant regions. Our evaluation demonstrates that our model improves upon traditional Naive Bayes classification by 47% and hierarchical TF-IDF by 27%. We further highlight the differences and commonalities with human reasoning about what is locally characteristic for a neighborhood, distilled from ten interviews and a survey that covered themes such as time, events, and prior regional knowledge.

Photos are an important information carrier for implicit relationships. In this article, we introduce an image based social network, called CelebrityNet , built from implicit relationships encoded in a collection of celebrity images. We analyze the social properties reflected in this image-based social network and automatically infer communities among the celebrities. We demonstrate the interesting discoveries of the CelebrityNet. We particularly compare the inferred communities with human manually labeled ones and show quantitatively that the automatically detected communities are highly aligned with that of human interpretation. Inspired by the uniqueness of visual content and tag concepts within each community of the CelebrityNet, we further demonstrate that the constructed social network can serve as a knowledge base for high-level visual recognition tasks. In particular, this social network is capable of significantly improving the performance of automatic image annotation and classification of unknown images.

Photos are becoming prominent means of communication online. Despite photos' pervasive presence in social media and online world, we know little about how people interact and engage with their content. Understanding how photo content might signify engagement, can impact both science and design, influencing production and distribution. One common type of photo content that is shared on social media, is the photos of people. From studies of offline behavior, we know that human faces are powerful channels of non-verbal communication. In this paper, we study this behavioral phenomena online. We ask how presence of a face, it's age and gender might impact social engagement on the photo. We use a corpus of 1 million Instagram images and organize our study around two social engagement feedback factors, likes and comments. Our results show that photos with faces are 38% more likely to receive likes and 32% more likely to receive comments, even after controlling for social network reach and activity. We find, however, that the number of faces, their age and gender do not have an effect. This work presents the first results on how photos with human faces relate to engagement on large scale image sharing communities. In addition to contributing to the research around online user behavior, our findings offer a new line of future work using visual analysis.

How do people keep track of their money? In this paper we present a preliminary scoping study of how 14 individuals in the San Francisco Bay Area earn, save, spend and understand money and their personal and family finances. We describe the practices we developed for exploring the sensitive topic of money, and then discuss three sets of findings. The first is the emotional component of the relationship people have with their finances. Second, we discuss the tools and processes people used to keep track of their financial situation. Finally we discuss how people account for the unknown and unpredictable nature of the future through their financial decisions. We conclude by discussing the future of studies of money and finance in HCI, and reflect on the opportunities for improving tools to aid people in managing and planning their finances.

Online recommendation communities, like Yelp, are valuable information sources for people. Yet, we assert, review communities have their own dynamics behind the social interactions therein. In this work, we study the Yelp review votes of useful, funny, and/or cool to understand these social perceptions of the review. We examine the relationship between these social signals and the emotional valence of the review itself (text and rating). We aim to understand the community's perception of each of these signaling contributions. We construct a conditional inference tree of social signals from 230K Yelp reviews to study how social signals shape the deviance in review rating from the mean rating, an indicator of the overall business rating on Yelp. We find two effects of social signals. First, reviews voted as useful and funny are associated with lower user ratings and relatively negative tone in the review text. Second, reviews voted as cool tend to have a relatively positive tone and higher ratings. Our findings open a research direction for further understanding of perceptions of social signals and have implications for design of recommendation systems.

Multimedia social communication is becoming commonplace. Television is becoming smart and social; media sharing applications are transforming the way we converse and recall events and videoconferencing is a common application on our computers, phones, tablets and even televisions. The confluence of computer-mediated interaction, social networking, and multimedia content are radically reshaping social communications, bringing new challenges and opportunities. This workshop, in its third edition, provides an opportunity to explore socially-aware multimedia, in which the social dimension of mediated interactions between people are considered to be as important as the characteristics of the media content. Even though this social dimension is implicitly addressed in some current solutions, further research is needed to better understand what makes multimedia socially-aware.

The Arts and Digital Culture program has offered a high quality forum for the presentation of interactive and arts-based multimedia applications at the annual ACM Multimedia conference for over a decade. This tutorial will explore the evolution of this program as a guide to new authors considering future participation in this program. By surveying both past technical and past exhibited contributions, this tutorial will offer guidance to artists, researchers and practitioners on success at this multifaceted, interdisciplinary forum at ACM Multimedia.

People take photos all over the world at all times of day; each photo depicting a place and a moment worth capturing. In the context of multimedia analysis and social computing, accurate location and time information about where and when these photos were taken is of importance for understanding event semantics, image content and many other purposes. While location information associated with photos is known to be relatively accurate, time is not. From a sample of 10 million public Flickr photos, we observe that 37% of the photos differ more than an hour between their camera timestamps and GPS timestamps with respect to local time at the locations where the photos were taken. Erroneous time information may adversely influence the correctness of any kind of temporal analysis that relies on camera timestamps, as well as research and real-world applications that require accurate knowledge of when and where photos were captured. In light of our observations we propose a simple yet effective metadata-only technique for improving the accuracy of camera timestamps.

Recently, the popularity of user generated videos has highlighted efficient video indexing and browsing as an urgent problem. Points of interest (POI) detection is a technique to address this issue by establishing the implicit relationship among different media resources. The majority of existing studies detect POI by visual similarity, leveraging computer vision techniques. However, these methods suffer from high computational complexity when processing large-scale video sets and are challenging due to the sparse visual correlation among different consumer videos. The advent of geo-referenced videos provides an opportunity to detect POIs in an efficient manner. In this work, we first propose a probability model to formulate the capture intention distribution for video frames. Second, we detect the POIs by considering the contributions from multiple videos. Evaluations demonstrate that our algorithm successfully detects POIs with a reduced error distance from several dozens to a few meters.

Multimedia social communication is becoming commonplace. Television is becoming smart and social; media sharing applications are transforming the way we converse and recall events and videoconferencing is a common application on our computers, phones, tablets and even televisions. The confluence of computer-mediated interaction, social networking, and multimedia content are radically reshaping social communications, bringing new challenges and opportunities. This workshop, in its second edition, provides an opportunity to explore socially-aware multimedia, in which the social dimension of mediated interactions between people are considered to be as important as the characteristics of the media content. Even though this social dimension is implicitly addressed in some current solutions, further research is needed to better understand what makes multimedia socially-aware.

New multimedia applications, such as community-created video repositories and tools for synchronous sharing, have revolutionized the ways that media is watched and shared. Effective instrumentation of these applications can enable researchers and system designers to better understand how video is being consumed: that is, how it is being watched, shared, augmented with annotations and otherwise experienced by individuals, by groups, and crowds. In addition to consuming content, however, people also talk about it. Conversational exchanges around video content can take place within applications (e.g., in chat spaces) and/or using separate communication channels (e.g., microblogs). We propose that the actions of these video viewers, with the video object itself and/or with each other around the video object, provide rich data for understanding the semantics and social relevance of various pieces of video content. We illustrate this approach with current research and a novel taxonomy of social multimedia interaction.

Multimedia social communication is filtering into everyday use. Videoconferencing is appearing in the living room and beyond, television is becoming smart and social, and media sharing applications are transforming the way we converse and recall events. The confluence of computer-mediated interaction, social networking, and multimedia content are radically reshaping social communications, bringing new challenges and opportunities. This workshop provides an opportunity to explore socially-aware multimedia, in which the social dimension of mediated interactions between people are considered as important as the characteristics of the media content. Even though this social dimension is implicitly addressed in some current solutions, further research is needed to better understand what makes multimedia socially-aware. In other words, social interactivity needs to become a first class citizen of multimedia research.

No abstract available.

Categorization of online videos is often treated as a tag suggestion task; tags can be generated by individuals or by machine classification. In this paper, we suggest categorization can be determined socially, based on people's interactions around media content without recourse to metadata that are intrinsic to the media object itself. This work bridges the gap between the human perception of genre and automatic categorization of genre in classifying online videos. We present findings from two internet surveys and from follow-up interviews where we address how people determine genre classification for videos and how social framing of video content can alter the perception and categorization of that content. From these findings, we train a Naive Bayes classifier to predict genre categories. The trained classifier achieved 82% accuracy using only social action data, without the use of content or media-specific metadata. We conclude with implications on how we categorize and organize media online as well as what our findings mean for designing and building future tools and interaction experiences.

A microblogged stream is delivered over time, providing an ongoing commentary of topics, trends, and issues. In this article, we present two methods of finding temporal topics within these Twitter streams. Using a normalized term frequency, we demonstrate how an effective table of contents can be extracted by finding localized "peaky topics". Second, we find "persistent conversations" which have a lower general salience but sustain and persist over the tweet corpus, in effect the whispering conversation that lingers in the background. These methods are demonstrated on a Twitter corpus of 53,000 tweets and a second Twitter corpus of 1.1 million tweets; the methods are generalizable to apply to any normalized scoring metric across a temporal corpus. We propose our method's implications on social media research and systems from a textual and social network analysis perpective.

With the proliferation of app stores and the advancement of mobile devices, research that might have only been tested with a dozen participants in the past can now be released to millions. This offers huge opportunities, but also requires adaptations of existing methods in dealing with large deployments and making sense of large data sets. This workshop provides a forum for researchers to exchange experiences and strategies for wide distribution of applications as well as gathering and analyzing large scale data sets.

In this article we describe the creation and exhibit of a participatory installation performance. Graffiti Dance allows the audience to graffiti paint with light onto a buildings side and receive immediate local feedback from a set of dancers choreographed to respond to the movement on the public display. The installation is a holistic experience using a plurality of sources (syndicated news Images and Twitter) and local influences (from mobile uploads) that reflect our understanding of the world around us, how we speak out in public forums, and how we interpret the creative act. We present the results of the performance from the perspective of the audience and the dancers and present new directions for future performances.

Television broadcasters are beginning to combine social micro-blogging systems such as Twitter with television to create social video experiences around events. We looked at one such event, the first U.S. presidential debate in 2008, in conjunction with aggregated ratings of message sentiment from Twitter. We begin to develop an analytical methodology and visual representations that could help a journalist or public affairs person better understand the temporal dynamics of sentiment in reaction to the debate video. We demonstrate visuals and metrics that can be used to detect sentiment pulse, anomalies in that pulse, and indications of controversial topics that can be used to inform the design of visual analytic systems for social media events.

This panel discusses how polling in the HTTPd protocol affects how we are building the next generation of the web and its applications. As other technologies (HTML, Javascript, etc.) move forward, we ask should the web's protocol also evolve or is it sufficient for the web to continue through just GET and POST?

This workshop is aimed at describing the elusive creative process: addressing models and of creative practice, from art to craft, from dance to education. In particular, we wish to discuss creative models that are conversational: connect the creator and the consumer via the creative act or artifact. The goal is to foster creative collaboration across domains and address the practice and design of the creative act to bring new ideas for researchers and practitioners alike.

Graffiti Dance creates a collaborative space for the voice of a local community to express a statement about the world around them. Several participants become graffiti artists, painting with an arsenal of lights via virtual graffiti mechanisms. Colors, images, and terms from popular sources (syndicated news Images & Twitter) and local influences (from mobile uploads) are suggested for use in the graffiti. Participants and viewers watch the graffiti's creation and receive feedback provided by a set of dancers who interact with the virtual (light) and physical space. Graffiti Dance presents a holistic experience using a plurality of sources that reflects our understanding of the world around us, how we speak out in public forums, and how we interpret the creative act.

Blogs, wikis, tagging, podcasts, and social networking websites such as MySpace, Facebook, Flickr and YouTube have radically changed user interactions on the World Wide Web from a static, one-way, consumption model to a dynamic, multi-way, participation model. Broad user power and flexibility have changed how people engage in and experience their interconnections, interests, and collaborations. Online social interactions will evolve in the next decade to address the growing needs of its user community and make entries into many aspects of our lives. This evolution may very well be among the most exciting ones of our times where the individual and collective power of people to contribute and share content, experiences, ideas, expertise etc. may be even more enhanced than it is today. The enhancement may be shaped through a better understanding of user needs and behavior and it may be enabled through the seamless convergence of multi-modal technologies, new applications, new domains, data mining and better navigational and search capabilities. Some of these changes will also permeate into the workplace and change the way we work. This panel will discuss how online social interactions may evolve in the next decade and what impact it may have on diverse dimensions in our world.

In this workshop, we describe and expose the mysterious creative process. We discuss models (both classical and contemporary) of creative practice and experience and their potential application to new media arts and technology. Models that connect the roles of creator and viewer/participant are of particular interest. The goal is to catalyze new ideas and foster creative collaboration across disciplines.

This paper describes a digital theater installation called Buzz. Buzz consists of virtual actors who express the collective voice generated by weblogs (blogs). These actors find compelling stories from blogs and perform them. In this paper, we explore what it means for a story to be compelling and describe a set of techniques for retrieving compelling stories. We also outline an architecture for high level direction of a performance using Adaptive Retrieval Charts (ARCs), allowing a director-level of interaction with the performance system. Our overall goal in this work is to build a model of human behavior on a new foundation of query formation, information retrieval and filtering.

In this paper, we describe MusicStory, a system that automatically creates videos to accompany music with lyrics. MusicStory uses common search engines, photo-sharing websites, and simple analysis of the dynamics and tempo of the music to create personalized photo-narratives. Video pacing and content is based on the content of the song and structure of the image repositories selected. The image associations MusicStory presents amplify the emotional experience by externalizing the imagery in song lyrics with the content found within a social network. The resulting work juxtaposes the meanings inherent in the social network with those in the song.

In this article, we present the Association Engine, a multimedia installation that explores the space of language and exposes connections between words by performing a series of improvisational games. It stands out from previous efforts in the realm of computers and theater as the agency has been empowered with a cultural understanding, more specifically, it uses the Web as a source of cultural reality.

In this article, we explore the structure of the web as an indicator of popular culture and its use in multi-media exhibits. In a series of art and technology installations, the software agency needs to keep 'grounded' to what people can readily understand. We administered a survey to understand how people perceived word and phrase obscurity related with frequency information gathered from a popular Web search engine. We found the frequency data gathered from the engine closely matched judgments gathered from people. The results of this study point to the new applications of the WWW in art and multi-media exhibits as an indicator of popular culture.

This article describes the implementation of a system that 'imagines' while a movie is being played by finding associations in the movie's content and presenting them to the viewer. This related information, in the form of images and movie clips, helps enhance the viewer's experience in a new immerse environment.