Crowdsourcing can collect many diverse ideas by prompting ideators individually, but this can generate redundant ideas. Prior methods reduce redundancy by presenting peers’ ideas or peer-proposed prompts, but these require much human coordination. We introduce Directed Diversity, an automatic prompt selection approach that leverages language model embedding distances to maximize diversity. Ideators can be directed towards diverse prompts and away from prior ideas, thus improving their collective creativity. Since there are diverse metrics of diversity, we present a Diversity Prompting Evaluation Framework consolidating metrics from several research disciplines to analyze along the ideation chain — prompt selection, prompt creativity, prompt-ideation mediation, and ideation creativity. Using this framework, we evaluated Directed Diversity in a series of a simulation study and four user studies for the use case of crowdsourcing motivational messages to encourage physical activity. We show that automated diverse prompting can variously improve collective creativity across many nuanced metrics of diversity.

Our world is witnessing the on-going substantial increase in the number of multimodal physical and social sensors that are ubiquitously distributed and observing or reporting what is happening in their surroundings. These sensors provide massive amounts of spatio-temporal digital footprints which can be analyzed for various tasks such as event detection or situation awareness. However, inherent noise due to the nature of these sensors result in imprecise data and hence imprecise analysis. Also, the heterogeneous data from different modalities, formats and sources make interpreting different levels of information a big challenge. To overcome these limitations, we propose a novel unified matrix factorization-based model to fuse physical and social sensor signals for spatio-temporal analysis. Readings of physical sensor signals are represented by a spatio-temporal situation matrix, which then incorporates social content that can provide explanations for the signal strengths. We test our framework on large-scale real-world data including PSI stations data, traffic CCTV camera images, and tweets for situation prediction as well as for filtering noise to detect events of diverse situations. The experimental results suggest that the proposed matrix factorization approach can utilize the sources correlation, resulting in better performances in various situational understanding tasks.

Interpretable machine learning models trade -off accuracy for simplicity to make explanations more readable and easier to comprehend. Drawing from cognitive psychology theories in graph comprehension, we formalize readability as visual cognitive chunks to measure and moderate the cognitive load in explanation visualizations. We present Cognitive-GAM (COGAM) to generate explanations with desired cognitive load and accuracy by combining the expressive nonlinear generalized additive models (GAM) with simpler sparse linear models. We calibrated visual cognitive chunks with reading time in a user study, characterized the trade-off between cognitive load and accuracy for four datasets in simulation studies, and evaluated COGAM against baselines with users. We found that COGAM can decrease cognitive load without decreasing accuracy and/or increase accuracy without increasing cognitive load. Our framework and empirical measurement instruments for cognitive load will enable more rigorous assessment of the human interpretability of explainable AI.

Participation on social media platforms has many benefits but also poses substantial threats. Users often face an unintended loss of privacy, are bombarded with mis-/disinformation, or are trapped in filter bubbles due to over-personalized content. These threats are further exacerbated by the rise of hidden AI-driven algorithms working behind the scenes to shape users' thoughts, attitudes, and behaviour. We investigate how multimedia researchers can help tackle these problems to level the playing field for social media users. We perform a comprehensive survey of algorithmic threats on social media and use it as a lens to set a challenging but important research agenda for effective and real-time user nudging. We further implement a conceptual prototype and evaluate it with experts to supplement our research agenda. This paper calls for solutions that combat the algorithmic threats on social media by utilizing machine learning and multimedia content analysis techniques but in a transparent manner and for the benefit of the users.

Search engines are still the most common way of finding information on the Web. However, they are largely unable to provide satisfactory answers to time- and location-specific queries. Such queries can best and often only be answered by humans that are currently on-site. Although online platforms for community question answering are very popular, very few exceptions consider the notion of users’ current physical locations. In this article, we present CloseUp, our prototype for the seamless integration of community-driven live search into a Google-like search experience. Our efforts focus on overcoming the defining differences between traditional Web search and community question answering, namely the formulation of search requests (keyword-based queries vs. well-formed questions) and the expected response times (milliseconds vs. minutes/hours). To this end, the system features a deep learning pipeline to analyze submitted queries and translate relevant queries into questions. Searching users can submit suggested questions to a community of mobile users. CloseUp provides a stand-alone mobile application for submitting, browsing, and replying to questions. Replies from mobile users are presented as live results in the search interface. Using a field study, we evaluated the feasibility and practicability of our approach.

Social media platforms are turning into important news sources since they provide real-time information from different perspectives. However, high volume, dynamism, noise, and redundancy exhibited by social media data make it difficult to comprehend the entire content. Recent works emphasize on summarizing the content of either a single social media platform or of a single modality (either textual or visual). However, each platform has its own unique characteristics and user base, which brings to light different aspects of real-world events. This makes it critical as well as challenging to combine textual and visual data from different platforms. In this article, we propose summarization of real-world events with data stemming from different platforms and multiple modalities. We present the use of a Markov Random Fields based similarity measure to link content across multiple platforms. This measure also enables the linking of content across time, which is useful for tracking the evolution of long-running events. For the final content selection, summarization is modeled as a subset selection problem. To handle the complexity of the optimal subset selection, we propose the use of submodular objectives. Facets such as coverage, novelty, and significance are modeled as submodular objectives in a multimodal social media setting. We conduct a series of quantitative and qualitative experiments to illustrate the effectiveness of our approach compared to alternative methods.

In the offline world, getting to know new people is heavily influenced by people’s physical context, that is, their current geolocation. People meet in classes, bars, clubs, public transport, and so on. In contrast, first-generation online social networks such as Facebook or Google+ do not consider users’ context and thus mainly reflect real-world relationships (e.g., family, friends, colleagues). Location-based social networks, or second-generation social networks, such as Foursquare or Facebook Places, take the physical location of users into account to find new friends. However, with the increasing number and wide range of popular platforms and services on the Web, people spend a considerable time moving through the online worlds. In this article, we introduce cyber-physical social networks (CPSN) as the third generation of online social networks. Beside their physical locations, CPSN consider also users’ virtual locations for connecting to new friends. In a nutshell, we regard a web page as a place where people can meet and interact. The intuition is that a web page is a good indicator for a user’s current interest, likings, or information needs. Moreover, we link virtual and physical locations, allowing for users to socialize across the online and offline world. Our main contributions focus on the two fundamental tasks of creating meaningful virtual locations as well as creating meaningful links between virtual and physical locations, where “meaningful” depends on the application scenario. To this end, we present OneSpace, our prototypical implementation of a cyber-physical social network. OneSpace provides a live and social recommendation service for touristic venues (e.g., hotels, restaurants, attractions). It allows mobile users close to a venue and web users browsing online content about the venue to connect and interact in an ad hoc manner. Connecting users based on their shared virtual and physical locations gives way to a plethora of novel use cases for social computing, as we will illustrate. We evaluate our proposed methods for constructing and linking locations and present the results of a first user study investigating the potential impact of cyber-physical social networks.

Recent years have seen a significant increase in the number of sensors and resulting event related sensor data, allowing for a better monitoring and understanding of real-world events and situations. Event-related data come from not only physical sensors (e.g., CCTV cameras, webcams) but also from social or microblogging platforms (e.g., Twitter). Given the wide-spread availability of sensors, we observe that sensors of different modalities often independently observe the same events. We argue that fusing multimodal data about an event can be helpful for more accurate detection, localization and detailed description of events of interest. However, multimodal data often include noisy observations, varying information densities and heterogeneous representations, which makes the fusion a challenging task. In this paper, we propose a hybrid fusion approach that takes the spatial and semantic characteristics of sensor signals about events into account. For this, we first adopt the concept of an image-based representation that expresses the situation of particular visual concepts (e.g. "crowdedness", "people marching") called Cmage for both physical and social sensor data. Based on this Cmage representation, we model sparse sensor information using a Gaussian process, fuse multimodal event signals with a Bayesian approach, and incorporate spatial relations between the sensor and social observations. We demonstrate the effectiveness of our approach as a proof-of-concept over real-world data. Our early results show that the proposed approach can reliably reduce the sensor-related noise, locate the event place, improve event detection reliability, and add semantic context so that the fused data provides a better picture of the observed events.

Location-based services gained much popularity through providing users with helpful information with respect to their current location. As a recent trend, virtual location-based services consider webpages or sites associated as 'virtual locations' that online users can visit. The presence of links between virtual locations and the corresponding physical locations (e.g., geo-location information of a restaurant linked to its website), allows for novel types of services and applications which constitute virtual location-based services (VLBS). Their success largely depends on the existence of websites referring to physical locations. In this paper, we investigate the usefulness of linking virtual and physical locations. For this, we analyze the presence and distribution of virtual locations, i.e., websites referring to places, for two Irish cities. Using simulated tracks based on a user movement model, we investigate how mobile users move through the Web as virtual space. Our results show that virtual locations are omnipresent in urban areas, and that the situation that a user is close to even several such locations at any time is rather the normal case instead of the exception.

Despite the services of sophisticated search engines, there are interesting information sources which are useful but largely inaccessible to current web users. These sources are often ad-hoc, location-specific and only useful for users over short periods of time, or relate to tacit knowledge of users or crowds. The solution presented in this paper introduces an integrated concept of "location" and "presence" across the physical and virtual worlds enabling ad-hoc socializing of users looking for similar information. While the definition of presence in the physical world is straightforward their definitions in the virtual world are neither obvious nor trivial. We provide an integrated spatial model spanning both worlds which enables us to define presence of users in a unified way. This integrated model allows us to enable ad-hoc socializing of users browsing the Web with users in the physical world specific to their joint information needs and allows us to unlock the untapped information sources mentioned above. We describe our proof-of-concept implementation and provide an empirical analysis based on real-world experiments.

The investigation of the browsing behavior of users has been a topic of active research since the Web started. However, new online services changed the meaning behind "browsing the Web" and require a fresh look at the problem. Platforms such as YouTube or last. Fm have started to replace the traditional media channels (cinema, television, radio) and media distribution formats (CD, DVD, Blu-ray). Particularly social networks (e.g., Facebook) attracted whole new, particularly less tech-savvy audiences. Advances in mobile technologies made browsing "on-the-move" the norm and changed the user behavior, often being influenced by the user's location and context in the physical world. Commonly used datasets, such as web server access logs or search engines transaction logs, are inherently not capable of capturing the browsing behavior of users in all these facets. DOBBS (DERI Online Behavior Study) is an effort to create such a dataset in a non-intrusive, completely anonymous and privacy-preserving way. DOBBS provides a browser add-on which keeps track of users' browsing behavior. In this paper, we outline the motivation behind DOBBS, describe the add-on and dataset, and present some first results to highlight the strengths of DOBBS.

Finding relevant and reliable information on the web is a non-trivial task. While internet search engines do find correct web pages with respect to a set of keywords, they often cannot ensure the relevance or reliability of their content. An emerging trend is to harness internet users in the spirit of Web 2.0, to discern and personalize relevant and reliable information. Users collaboratively search or browse for information, either directly by communicating or indirectly by adding meta information (e.g., tags) to web pages. While gaining much popularity, such approaches are bound to specific service providers, or the Web 2.0 sites providing the necessary features, and the knowledge so generated is also confined to, and subject to the whims and censorship of such providers. To overcome these limitations we introduce COBS, a browser-centric knowledge repository which enjoys the inherent openness (similar to Wikipedia) while aiming to provide end-users the freedom of personalization and privacy by adopting an eventually hybrid/p2p back-end. In this paper we first present the COBS front-end, a browser add-on that enables users to tag, rate or comment arbitrary web pages and to socialize with others in both a synchronous and asynchronous manner. We then discuss how a decentralized back-end can be realized. While Distributed Hash Tables (DHTs) are the most natural choice, and despite a decade of research on DHT designs, we encounter several, some small, while others more fundamental shortcomings that need to be surmounted in order to realize an efficient, scalable and reliable decentralized back-end for COBS. To that end, we outline various design alternatives and discuss qualitatively (and quantitatively, when possible) their (dis-)advantages. We believe that the objectives of COBS are ambitious, posing significant challenges for distributed systems, middleware and distributed data-analytics research, even while building on the existing momentum. Based on experiences from our ongoing work on COBS, we outline these systems research issues in this position paper.

Improving web search solely based on algorithmic refinements has reached a plateau. The emerging generation of searching techniques tries to harness the ``wisdom of crowds'', using inputs from users in the spirit of Web 2.0. In this paper, we introduce a framework facilitating friends augmented search techniques (FAST). To that end, we present a browser add-on as front end for collaborative browsing and searching, supporting synchronous and asynchronous collaboration between users. We then describe the back end, a distributed key-value store for efficient information retrieval in the presence of an evolving knowledge base. The mechanisms we explore in supporting efficient query processing for FAST are applicable for many other recent Web 2.0 applications that rely on similar key-value stores. The specific collaborative search tool we present is expected to be an useful utility in its own right and spur further research on friends augmented search techniques, while the data-management techniques we developed are of general interest and applicability.

In open environments, deciding if an individual is trustworthy, based on his past behavior, is fundamentally important. To accomplish this, centrality in a so-called feedback graph is often used as a trust measure. The nodes of this graph represent the individuals, and an edge represents feedback that evaluates a past interaction. In the open environments envisioned where individuals can specify for themselves of how to derive their trust in others, we observe that several centrality computations take place at the same time. With centrality computation being an expensive operation, performance is an important issue. While techniques for the optimization of a single centrality computation exist, little attention so far has gone into the computation of several centrality measures in combination. In this paper, we investigate how to compute several centrality measures at the same time efficiently. We propose two new optimization techniques and demonstrate their usefulness experimentally both on synthetic and on real-world data sets.

User-generated content in the spirit of Web 2.0 is a promising means to improve the search for relevant information on the web, particularly multimedia content. The idea is that user col-laboratively search or browse for information, either directly by communicating or indirectly by adding meta information (e.g., tags) to web pages. However, current solutions are bound to specific web sites providing such features. To overcome this limitation we introduce COBS, making Web 2.0 features available also for the 'old' web. In this demo we present the front-end, a browser add-on that enables users to tag, rate or comment arbitrary web pages and to communicate with others in both a synchronous and asynchronous manner.